Activity 9
Playing with Notes
I look forward to this activity so much because it involves music!!! When I was younger, my mother always drag me and my siblings to singing contests and talent shows. If my course is not physics, I'd be a theater major.
This activity let's make SCILAB SING SING SIIIIIIIINNNGGGG! To do this we, need to integrate a lot of the things we've learned from the past activities.
By the way, I feel so refreshed today for I finished blogging the dreaded and long Activity 8 on Morphological Operations.
Let me first discuss some music notation basics. In a stanza, there are five lines. Letters are used to denote the placement of the notes in the stanza. A sample code that makes SCILAB sing Mary had a little lamp is illustrated below:
The values of the notes are frequency values that I got from [4]. For this activity, I chose a very simple score. My childhood favorite, Twinkle, Twinkle, Little Star. The score is shown below:
I then cropped the stanzas in GIMP:
cropped stanza 1 and 2
It is easier for me to manipulate images with a black background such that the blobs will have the highest intensity value. Using the SegmentByThreshold() function in SCILAB, I binarized the image with a threshold value of 100.
black and white thr < 100
Next, using my vast knowledge on morphological operations thanks to Activity 8, I opened the image with a vertical line 3 pixels thick thus removing horizontal profiles as shown:
clean 1
For the music that I am trying to recreate, I am only considering full notes. So my next step is to close holes and what better operation to use other than the closing operator. I used a circular structuring element with radius 6 resulting to:
clean 2
The next step is to remove the vertical lines by doing an opening operation with a horizontal line as a structuring element:
clean 3
Finally to make the blobs roundish, I open the image again but this time by using a circle with radius 6 as a structuring element:
clean 4
The next thing that I need to do is to determine the note by virtue of its vertical position. I imported the image in GIMP and identified the ranges (with corresponding frequency values) as:
C = 120 - 145 [261.63*2]
G = 77 - 107 [392.00*2]
A = 63 - 95 [440.00*2]
F = 89 - 115 [349.23*2]
E = 97 - 125 [329.63*2]
D = 108 - 135 [293.66*2]
By using SearchBlobs() and AnalyzeBlobs(), I calculated the centroid of each blob. After SearchBlobs(), the image looks like this:
When the centroid of the blob falls under a specific range of a note, it plays that note. I saved the output as an mp3 file, converted it to video using [2] and uploaded the video in YouTube. I only generated a music file for the first stanza. Feel free to listen.
The reconstruction wasn't at all perfect yet since I approximated (or closed) the half note to a full note.
My take away in this activity is that it really takes talent to do morphological operations. To further improve the quality of the reconstruction a more accurate method should be devised to depict the range of the note. My method of manually identifying in GIMP was not at all accurate.
I would give myself a rating of 12/10 for producing all the required images. I would like to thank wix.com for their very nice website platform.
References:
[1] M. Soriano. “A9 - Playing Notes by Image Processing,” Applied Physics 186 Activity Hand-outs, 2016.
[2] tovid.io [for converting my mp3 to video with a photo]
[3] singing-bell.com http://www.singing-bell.com/wp-content/uploads/2014/12/TwinkleTwinkle_C_Image.jpg
[4] Physics of Music - Notes. http://www.phy.mtu.edu/~suits/notefreqs.html