"Set instrument" to one of the sprite's sounds?

[ericrosenbaum]

In addition to the built-in instruments (such as piano, flute, etc), the "set instrument" block could allow you to choose one of the sounds in the sprite. Then, the "play note" block would play this sound at different pitches by changing its playback rate. "Play note 60" (middle C) would play the sound unchanged, "play note 61" would shift it up a semitone, etc. The duration set by the "play note" block would stop the sound if necessary, with a short fade out to prevent clicks. So, for example, you could record your dog barking, and make it bark a melody, like happy birthday.

Early experiments with this idea are promising. Seems like fun.

One concern is that the pitch of the sounds produced in this way will depend on the pitch of the recorded sound, which could be confusing, especially if you are trying to use multiple sounds together, or in combination with the built in instruments.

Design questions:

• What should the menu look like? We'll need two sections of the menu, one for the built-in instruments, and one listing the sounds in the sprite.
• How does selecting an instrument by index work? i.e. currently you can select piano by setting the instrument to 1. How would you select the first sound in the sprite by number?
• One solution would be to modify the "set instrument" block so it has an additional menu, to choose between sprite sounds and instruments.

[ericrosenbaum]

Another option for this would be to have a second "set instrument" block, with a menu that lists the sounds in the sprite. So by default for the cat sprite it would say "set instrument to [meow]". You could drop a reporter onto that menu to select sounds by index (as in the play sound block).

[ed-cooper]

In the scratch vm we now have the set effect [pitch] to (100) block, so adding this wouldn't give any real additional flexibility to Scratch 3.
(For most scratchers at least, maybe some might try to precisely tune sounds to notes which this would help with, but I figure eventually a conversion could be figured out from pitch to note, and it's such a rare use case any implementation time isn't really worth it)

[towerofnix]

I'm partially agreeing with chooper100, but - setting the instrument to a sound really does encourage using sounds as audio samples; it almost seems a bit like a "hack" to need to use a mapping of notes to pitches. So, no, it doesn't add any flexibility, but I'm wondering if the amount of simplicity it'd mean to implement this (on the user end, of course) counters that enough..

[ed-cooper]

Also, I could imagine a lot of younger scratchers getting confused over why their sounds are turning up in the instruments block

[ericrosenbaum]

Thanks for the comments on this idea. It's a good observation that the effect of playing a sound as an instrument would be similar to using the pitch effect block. There are some differences though. One is that sounds played with the play note block can be polyphonic: you can have multiple copies of the sound playing simultaneously, and at different pitches (using play sound and pitch effect, this would require using multiple sprites or clones). So you could play a major chord with a meow as easily as with a piano. Another difference is that the play note block lets you pick a duration (which would make a longer sound stop early), so you can make melodies with rhythms that work with other play note and play drum blocks.

For reference, changing the pitch effect by 10 causes a change of 1 semitone. So, for example, to change the pitch by an octave, or 12 semitones, you would set the pitch effect to 120.

[SharkPool-SP]

Thanks for the comments on this idea. It's a good observation that the effect of playing a sound as an instrument would be similar to using the pitch effect block. There are some differences though. One is that sounds played with the play note block can be polyphonic: you can have multiple copies of the sound playing simultaneously, and at different pitches (using play sound and pitch effect, this would require using multiple sprites or clones). So you could play a major chord with a meow as easily as with a piano. Another difference is that the play note block lets you pick a duration (which would make a longer sound stop early), so you can make melodies with rhythms that work with other play note and play drum blocks.

For reference, changing the pitch effect by 10 causes a change of 1 semitone. So, for example, to change the pitch by an octave, or 12 semitones, you would set the pitch effect to 120.

how about just adding blocks to start/stop sounds at inputted times. As well as reporters for the current time.
If it interests you, potentially you could take inspiration from a Turbowarp extension I made:
https://github.com/SharkPool-SP/SharkPools-Extensions/blob/main/extension-code/Tune-Shark-V3.js

To try it, load it in turbowarp as a unsandboxed custom extension. This is entirely possible in scratch with some audio engine tweaks