Plugin access to MIDI - Sight Reader?

• May 7, 2012 - 17:09

Can a JavaScript plugin take input from a MIDI keyboard?
I have in mind a tool to help with sight-reading. MuseScore would display the tune and at each stage the plugin would turn notes a different colour when they have been played. The plugin would not check tempo, just check that all the required notes are down, with no extras.

Comments

• May 7, 2012 - 18:49

No. Javascript plugins don't have access to MIDI keyboard currently. You could create an independent application and call MuseScore via OSC to highlight the notes though. See osc.cpp in the source code.

Reply

• May 7, 2012 - 20:03

Thanks.

For your suggestion to work, I'd need to get the tune out of MuseScore into my sight-reading program, then monitor the MIDI keyboard, match up the notes and send messages to MuseScore to colour in the correct notes.
I can handle the MIDI control and note matching, but can I get the tune out of MuseScore using osc?

I usually work in Java, not C++, so would I have to compile your osc.cpp into a DLL, then generate a Java wrapper that I can call from my Java code?
Or is it easier to just send bytes to the port 5282 directly from Java, perhaps using this: http://opensoundcontrol.org/implementation/java-osc?

Reply

• May 7, 2012 - 20:29

osc.cpp contains the list of commands that MuseScore can handle. You might need to happend some more here if you need to activate some features through OSC that are not listed yet.

You could add a command to save to MIDI in the osc command lists or call a plugin that will do so

Your java software could also be run by a MuseScore plugin. The user launches the plugin, the current score is saved as MIDI by the plugin (this is already possible), then the plugin launches your java software, your java software handle MIDI in and note matching, and it sends OSC message to MuseScore to do the coloring.

In MuseScore trunk, there is a osc command /color-notes it takes a tick,pitch argument and it will color the note in red.

Reply

• May 8, 2012 - 21:09

MIDI note numbers and midi ticks in the MIDI file you got by saving it in MuseScore. (nothing related with MusicXML).
The /color-notes is ugly... it's a string argument, comma separated. If I get a change, I will change it to two int...

Reply

• May 9, 2012 - 21:03

Thanks. This is starting to look workable.

I am working with MuseScore V1.1. Is this OK, or do I need V1.2, or the trunk from your Subversion?
I have written a short piece of test code using the Java OSC (http://opensoundcontrol.org/implementation/java-osc), but it has no effect. No errors reported. The MIDI file has the C an octave above middle C and also an E just above middle C at tick 0.

I would also like to be able to control the colour - could you add that as a parameter?

Here's my test code...

package com.dch.sightreader;

import java.net.InetAddress;

import com.illposed.osc.OSCMessage;
import com.illposed.osc.OSCPortOut;

public class SightReader
{
public static void test()
{
System.out.println("Starting...");
try
{
int museScoreOSCPort = 5282;
OSCPortOut sender = new OSCPortOut(InetAddress.getLocalHost(), museScoreOSCPort);
Object args[] = new Object[1];
int midiNote = 60+12; // C5
int midiTick = 0;
args[0] = "" + midiTick + "," + midiNote;
OSCMessage msg = new OSCMessage("/color-note", args);
sender.send(msg);

midiNote = 64; // E5
midiTick = 0;
args[0] = "" + midiTick + "," + midiNote;
msg = new OSCMessage("/color-note", args);
sender.send(msg);
}
catch (Exception e)
{
System.out.println("Couldn't send");
}
}

public static void main(String[] args)
{
test();
}
}

Reply

• May 9, 2012 - 21:50

There is no OSC in MuseScore 1.1. In MuseScore 1.2, it's hidden and MuseScore 1.2 codebase is different than the trunk. So it would be better to use a nightly build. See [[nodetitle:Comparison of stable, prerelease, and nightly builds]] .
OSC is off by default in nightlies. You need to go to Edit -> Preferences -> General to enable it and choose the OSC port. Then restart MuseScore. And of course, in your example, you need a score loaded with a C on tick 0 ;)

if you want to discuss in live chat you can join our IRC channel (#musescore on freenode.net)

Reply

• May 10, 2012 - 20:40

I downloaded the latest nightly build and followed your instructions.
My test code now colours a couple of notes red as expected; I need to do some more testing on it.

If you can add another param for note colour, I'd appreciate it.
I may have to ask for more advice on the plugin bit later, but I can get most of the Java bit written now.

Thanks.

Reply

• May 13, 2012 - 11:03

I've got a basic stand-alone sight reader working fine now.
Can I just modify osc.cpp to take a third (optional) colour parameter, or is this something you guys need to do?

Reply

• May 14, 2012 - 20:34

I got this when I tried to commit:
Error: access to '/svnroot/mscore/!svn/act/b650aec0-796b-da47-81a1-7eb36dc24d67' forbidden

My new version of the function is below, if you have time to commit it for me:

//---------------------------------------------------------
// oscColorNote
//---------------------------------------------------------
void MuseScore::oscColorNote(QString s)
{
if(!cs)
return;
QStringList args = s.split(",");
if(args.length() >= 2) {
int tick = args[0].toInt();
int pitch = args[1].toInt();
QColor noteColour = null;
if(args.length == 3){
noteColour = QColor(args[2]);
}
else{
// Default to red for compatibility with previous version.
noteColour = QColor("red");
}

Measure* measure = cs->tick2measure(tick);
if(!measure)
return;
Segment* s = measure->findSegment(SegChordRest, tick);
if (!s)
return;
//get all chords in segment...
int n = cs->nstaves() * VOICES;
for (int i = 0; i < n; i++) {
Element* e = s->element(i);
if (e && e->isChordRest()) {
ChordRest* cr = static_cast(e);
if(cr->type() == CHORD) {
Chord* chord = static_cast(cr);
for (int idx = 0; idx < chord->notes().length(); idx++) {
Note* note = chord->notes()[idx];
if (note->pitch() == pitch) {
cs->startCmd();
cs->undo(new ChangeProperty(note, P_COLOR, noteColour));
cs->endCmd();
cs->end();
return;
}
}
}
}
}
}
}

Reply

• May 21, 2012 - 17:00

Hi DonH, I added the feature of the optional color in r5657. I didn't use your patch and changed the API to something cleaner. You can now call /color-note with 2 int (tick and pitch) and a string (color) in a list.

Reply

• May 23, 2012 - 20:49

Thanks.

In the C++ you have the tick parameter as an int, but in Java-land it is a long.
Is this a mistake or will a C++ int hold a Java long? I'm using 32-bit Vista.

When I pass in an Integer, it works, but when I pass in a Long it does not.

Is it possible to display a message box from the JavaScript plugin code?

Reply

• May 29, 2012 - 08:20

Seems you copied over a mistake from one of multiple other plugins: your plugin too ignores voice 4.
In line 53:
for (var v = 0; v < 3; v++) // Iterate through the voices.

This got to be:
for (var v = 0; v < 4; v++) // Iterate through the voices.

See #16691: plugin 'MIDISightReader' does not work on voice 4

Also I guess you should mention that it only works on Windows and only with the Nightlies (and only as of r5657)?

Reply

Do you still have an unanswered question? Please log in first to post your question.