f2e-spec
f2e-spec copied to clipboard
alibaba
Pitchdetector2 - further tuning
This is an experimental branch that does more than #540 which itself can better be compared with the master branch. The changes in this one seem to greatly reduce the number of extra tones detected and shown on practice screen. More testing is still needed.
If all is good, I expect to merge these changes back to pitchdetector, which maintains cleaner history (mesonbuild is merged to this so that I could hack it).
Also... this is just a question/suggestion, since you're going elbow-deep into pitch processing, how do you think about adding a "Difficulty" setting like Ultrastar does? Is it easily doable?
Scoring and thus difficulty settings would be handled in an entirely different part than the pitch detector (I gather that might be in player.cc but you'll need to look it up). The idea has been discussed before, and there is not much use for less strict scoring because Performous already gives gradually less score with less accuracy (it is not on/off like in singstar). Not having difficulty levels keeps scores comparable, too.
Scoring and thus difficulty settings would be handled in an entirely different part than the pitch detector (I gather that might be in
player.ccbut you'll need to look it up). The idea has been discussed before, and there is not much use for less strict scoring because Performous already gives gradually less score with less accuracy (it is not on/off like in singstar). Not having difficulty levels keeps scores comparable, too.
I thought maybe it could go in here so long as it is using a less strict comparison of notes expected and sung (then again, perhaps that logic IS elsewhere)
Analyzer only gives out detected frequencies in Hz, it doesn't know about notes.
Analyzer only gives out detected frequencies in Hz, it doesn't know about notes.
Ah ok. So they get converted to notes/tones elsewhere?
Now it works much better than the old one. Clear and uberprecise detection in practice, and seems to be working in singing too. During testing I got one segfault but couldn't determine if it was caused by this patch or not.
@Lord-Kamina See Notes::powerFactor that's the essence of vocal scoring. MusicalScale for conversions between Hz and note numbers. Player::update for the main vocal grading loop.
Now it works much better than the old one. Clear and uberprecise detection in practice, and seems to be working in singing too. During testing I got one segfault but couldn't determine if it was caused by this patch or not.
Cool. Maybe post a trace or dump if you have it to see of we can figure it out?
This is still a work in progress. I'm developing the algorithm in Python, and what you see here is a port to C++ (curiously, the Python version runs faster). There are further developments done and more planned, that I plan to add here once complete. Unfortunately I'm still stuck to testing mostly without real microphone input, so any feedback from your experiments is more than welcome.
Converted to draft according to latest comment: "This is still a work in progress"
It might take a very long time until I can pick this up again. If you don't mind keeping around a draft, feel free to keep it here. Possibly some feedback of the other Pitchdetector PR inspires me to complete this work still this Summer.
This is also available for testing. IMO this has clearly better performance than either master or pitchdetector1 but YMMV because the algorithm is largely redesigned.
If there is agreement that this algorithm works well, I suggest merging this instead of pitchdetector1. If further development on this is needed, then pitchdetector1 should probably be merged meanwhile.
@Tronic this includes/is built on top of the changes from #540 right?
If that's the case @ooshlablu should test this instead and #540 can be closed
@Baklap4 Yes, this is intended to supersede the other PR. The new phase equation is included and levels are tuned in this as well. Also, if the other PR is merged, there will be a merge conflict with this, so in that regard it might be nice to close the other one and only merge this.
The problem is that this changes a lot more, while the other one was intended to be merged without much testing due to it being a very minimal change with little expected adverse effects. I suggest several developers do thorough testing in noisy and quiet environments and with different singers before merging this.
In particular, high-pitched (female) voice needs to be tested. And since whistling is supported as a new feature, one might try that as well (be sure not to blow on your mic but beside it).
And since whistling is supported as a new feature
"Always look on the bright side of life - Monty python" is a great song for whistling ;)
I'll test #540 anyway since I already built the package and it seems like it's easier to test by myself.
I will do a test of this myself at some point today and see if i can get a proper group to fully test this PR over the weekend. Finding someone that can actually whistle to a song might be difficult tho :-)
someone that can actually whistle to a song
It was surprisingly easy to hit the notes on Hard difficulty. The pitch wave looks different too.
EDIT: Forgot to mention that I only tested on Ubuntu 20.04
For me, this is really choppy and appears to be not picking up the pitch compared to how #540 and current master is.
I tried with 2 different mics on Behringer USB interface, trying different volume levels, and a straight USB mic, and had results that look like this:
I couldn't find where to tune in the settings, and the results are similar on all the difficulty levels, just thinner lines :)
If I had to pick between #540 and this, I would pick that to merge for now. Let me know what I can do to get you any info you need to help.
The pitch waves show high frequency, suggesting that something other than your singing (base frequency) is being detected. Could you check in practice screen visually which notes appear? Also check your volume level, should be around -20 dB while singing. If much louder, it probably cannot work, and more silent should produce no notes.
If your CPU is really show or you are on a non-optimised debug build, there could also be performance issues. I think the new algorithm is heavier than the old one but still it runs fine on my 2015 Macbook Pro with four mics, so I doubt this would have effect on any modern PC.
@Tronic I see what you are saying now. If I play with the output volume from Performous very low/off it seems more stable and less jumpy overall when singing along with a song. This does have major interference with the way I play tho, which is with a group rock-band style with 1 mic and guitar controllers, rowdy,and hence high volumes through high-powered, and very responsive speakers :-).
At the moment, I can't get this to work with any sort of outside noise. I've tried adjusting the USB mic volume in pulse audio, which I don't think has any effect since this uses SDL to interface devices directly (I could be wrong, but that's how it seems to me when messing with volumes), and I also have tried turning down the input gain from the mic through the Behringer interface. At very low gain through the Behringer, things do seem to smooth out, but there are "blanks" and other chops that are definitely the result of low sensitivity at such low volumes.
@Baklap4 I think for now #540 is the better candidate that does make a difference in terms of scoring and actual game play if this is something that should make it into a new release. I think we should mark the PRs appropriately.
@ooshlablu I think you need to adjust mic volume in alsamixer, switch to capture volumes on the physical device of the mic(s). If your device supports that.
Could you confirm which dB value you see in practice screen while singing? Does it ever reach the top (0 dB)?
1.2.0 is released a month ago. This was expected to be merged after this. Can we get this PR up-to-date and test it? @ooshlablu
@Tronic if you could look at the conflicts, the packages will build and get attached to this PR now. I can install it and test it again and see how it looks now.
Hmmh, that's some pretty nasty conflicts. I'll see if I can get this compiled and running with actual mics, to be able to merge with meaningful dB limits.
As an update, I am missing some cables needed for my karaoke setup, and thus will only be able to test this with Macbook internal mic. I suggest others also do some testing and possibly fiddling with the values but I'll try to get to my tests in the coming days too.
If there may be problems with the new pitch detection why not make it configurable?
