chords-db
chords-db copied to clipboard
Is the first C major correct?
I'm not an expert, but the first C major chord looks a bit strange to me:
frets: '332010',
fingers: '342010'
I usually play it like this:
frets: 'x32010',
fingers: '032010'
Your variant looks more like a 'C/G' to me. What do you think?
Yess, you're right. It's more standard the Cmajor that you say, I have seen it in multiple sources, some muted the 6th string and others prefers the G note in the 6th chord. Some others represented it as a variation of the same chord.
So you think it would be a C/G variation, very interesting. I'm going to change it like you say. Do you know any source where I could query the "slash" chords so I can add them?
I'm finishing a web visualization tool of all the chords so it would be easier to browse them and detect all the typos that I could make in them.
Thanks
I couldn't find a good source that includes slash chords and finger positions, unfortunately...
chordsdatabase.com
seems to have a pretty nice data base including finger positions, lots of variations for each chord and a lot of suffixes, but no slash chords. I wrote a little script to extract all the data. Maybe this this helps a bit: https://pastebin.com/raw/8kQK8Z37
I saved the data in a simple format like this:
chords: {
"C#":[{
"positions": ["x","4","6","6","6","4"],
"fingerings": [["0","1","3","3","3","1"], ["0","1","2","3","4","1"]]
},{
"positions": ["9","11","11","10","9","9"],
"fingerings":[["1","3","4","2","1","1"]]
}, ...],
"C#m":[{
"positions":["x","4","6","6","5","4"],
"fingerings":[["0","1","3","4","2","1"]]
},{
"positions":["9","11","11","9","9","9"],
"fingerings":[["1","3","4","1","1","1"]]
}, ...],
...
}
Keep in mind that there is a lot of redundancy in that file because every chord is present in its sharp and flat form (Ab == G#, E# == F, ....).
Hi @T-vK, are you still interested in this topic? I've just started to add some slash chords, just check this issue: https://github.com/tombatossals/chords-db/issues/6; Would you like to join us for extending the database?
@szaza Hi, I'm still very much interested. I just haven't had the time to work on it. I did find a pretty complete "database": https://jguitar.com/chordsearch?chordsearch=&labels=finger I think it truely contains every possible chord including fingering positions. Unfortunately it only shows one fingering possibility per shape, but nevertheless I think it would be worth it using the data. I'd just have to write a little script that scrapes the data for every possible chord. So basically I need a list of all possible chord names. Then it's just a question of time until all the data is scraped.
Edit: I think this page would make a good starting point for scraping: https://jguitar.com/chord?root=C&chord=Major&bass=C&labels=finger&gaps=0&fingers=4¬es=sharps
We could then simply have three nested loops. The first one iterating over the root note, the second one iterating over the chord variant and the third one iterating over the bass note. This would result in about 10000 HTTP requests. For some chords we need to send multiple requests as there are so many possible shapes that there are multiple pages.
Hi @T-vK, I'm very glad that you already started to find a way to obtain all the slash chords, however I'm not sure that fetching the jguitar.com is the best one. My doubts are the followings:
- we would need parseable resources (JSON format would be the best), however as I see jGuitar returns chords only as images (in .png format); Of course we could write an image processing algorithm, that converts images to JSON format, but that would take some time;
- also I do not consider parsing another site as the most ethical solution, maybe it would be better to contact them and request chords from them. As guitar chords are their real value, I don't think so that they will provide us the table of the chords for free, but maybe it worth a try;
- maybe we could find a way to generate all the available slash chords: by transposing base chords we can obtain new chords.
@tombatossals how did you generate the current database? Did you put down all the chords by your hand?
Processing the images wouldn't be that hard actually and would only be necessary to obtain the finger positions. The rest can be extracted from the URL to each image. I'm pretty sure they have an algorithm to generate all these chords rather than a database. I don't really see a problem with parsing their public content. I mean if you make something publicly available on the Internet then everybody can use it. I'd obviously give them credits, but since they don't sell that information and I certainly have no plans to do so either, I don't think it would be unethical at all.
Hi,
"Processing the images wouldn't be that hard actually and would only be necessary to obtain the finger positions." - Can we try it out first on a single image? What approach would you use for it? Have you done similar tasks?
"I'm pretty sure they have an algorithm to generate all these chords rather than a database" - yes, I'm also fully sure about it, however it is enough to generate the database only once, because guitar chords won't change by passing the time.
"I mean if you make something publicly available on the Internet then everybody can use it." - Yes, it is free to use, however I'm not sure that it is allowed to make a copy. Just consider using YouTube, it is free to use, but it is not permitted to download the content and put it available on another site.
In my opinion the right approach would be to write an algorithm that generates chords instead of spending time with writing image processing and site parsing algorithms.
Can we try it out first on a single image? What approach would you use for it? Have you done similar tasks?
Well testing it on one image first would be the first step for sure. I have done similar things. For instance I have written a proof of concept showing how easy it is to decipher a simple captcha. The only difference for the chord diagrams would be that I'd have to find the positions of every number found in the image. https://github.com/T-vK/crack-captcha-poc
In my opinion the right approach would be to write an algorithm that generates chords instead of spending time with writing image processing and site parsing algorithms.
I fully agree, but I know way too little about music theory to do something like that.
Just consider using YouTube, it is free to use, but it is not permitted to download the content and put it available on another site.
You can't really compare the two because you can't have a copyright on a chord. A fair comparison would be if someone uploaded a video that is in the public domain. In that case there are no laws that would disallow making copies of it.
@tombatossals how did you generate the current database? Did you put down all the chords by your hand?
Yesss, all by hand. I started it as a proof of concept and finished coding the chords by hand I found in some old books I have.
@tombatossals what do you think about @T-vK's approach? Can we give a try to obtain the missing chords from jGuitar?
I haven't had much time to get this to work. But here is what I ended up with on the weekend:
const Tesseract = require('tesseract.js')
const sharp = require('sharp')
async function imageToNumber(filePath) {
const result = await Tesseract.recognize(filePath, {
lang: 'eng',
tessedit_char_whitelist: '0123456789T' // allow letter T and numbers
})
console.log(result)
return parseInt(result.text)
}
const fingerWidth = 15
const fingerHeight = 15
const stringPositions = [18,43,68,93,118,143]
const fretPositions = [55,80,105,130,155]
const baseFretPositionX = 162
const baseFretPositionY = 51
const baseFretPositionWidth = 24
const baseFretPositionHeight = 24
async function main() {
const inputImg = await sharp('test.png')
let tmpOutput
tmpOutput = await inputImg.extract({ left: baseFretPositionX, top: baseFretPositionY, width: baseFretPositionWidth, height: baseFretPositionHeight }).toBuffer()
//await inputImg.extract({ left: baseFretPositionX, top: baseFretPositionY, width: baseFretPositionWidth, height: baseFretPositionHeight }).toFile('output.png')
const baseFret = await imageToNumber(tmpOutput)
console.log('Base fret:', baseFret)
const fingering = []
for ([i,string] of Object.entries(stringPositions)) {
let fingerFound
let fingerFoundFret
for ([j,fret] of Object.entries(fretPositions)) {
tmpOutput = await inputImg.extract({ left: string, top: fret, width: fingerWidth, height: fingerHeight }).toBuffer()
const finger = await imageToNumber(tmpOutput)
if (!isNaN(finger)) {
fingerFound = true
fingering.push(finger)
break
}
}
if (!fingerFound) {
fingering.push(0) // TODO: check if string is muted and push 'x' in that case
}
}
console.log('Fingering detected:', fingering)
process.exit()
}
main().catch(console.error)
Input:
Output:
Fingering detected: [ 1, 3, 4, 1, 1, 1 ]
So yeah, it basically works, but something is still wrong. It seems to detect a "1" when there is a "2".
It looks pretty good, maybe I'll have some time during this week to refine a little bit your implementation. We also have to calculate the position of the fingers on the frets, so in this case it would be: frets [1,3,3,2,1,1] and also barres: 11 is important.
Yes, I kinda forgot about that. But the information is already being extracted.
The variable baseFret
contains the fret number that is found on the right of the diagram.
Using the variable fret
in the for loops you can get the fret position of the fingers by calculating baseFret+fret
. The string can be extracted from the variable string
.
Edit: I just remembered, it is not really necessary, because that information can be extracted fromt he HTML code:
<img src="/images/chordshape/Dsharp-Major-Dsharp-11%2C13%2C13%2C12%2C11%2C11-sharps-finger.png" alt="D# chord {11 13 13 12 11 11} chord" class="img-responsive" width="200" height="200">
We can simply extract the {11 13 13 12 11 11}
part to get the positions.
So yeah, the main problem right now is that the "2" is being detected as a "1". Once that is solved, I could parse all the data.
Edit2:
Correction: To get the string and fret number inside the for loops you'd have to use the i
and j
actually. i
being the string (0-5) and j
being the relative fret number. The actual fret number would be baseFret+j
then.
Okay, I think I have everything I need now. I managed to get it to recognize the numbers correctly now. I've pretty much already written the whole code that is necessary to extract all the data. I just need to add error handling because with 10000 http request, some are bound to fail and I'd rather not start from scratch every time that happens.
My code so far:
#!/usr/bin/env node
const { createWorker, SetVariable } = require('tesseract.js') // npm i tesseract.js@next
const sharp = require('sharp')
const rp = require('request-promise-native')
const cheerio = require('cheerio')
const fs = require('fs-extra')
const download = require('download')
const digitWorker = createWorker({
//logger: m => console.log(m)
})
const numberWorker = createWorker({
//logger: m => console.log(m)
})
async function imageToNumber(image) {
/*const result = await Tesseract.recognize(image, {
lang: 'eng',
//tessedit_pageseg_mode: '10', // singe char mode
tessedit_char_whitelist: '0123456789T' // allow letter T and numbers
})
Tesseract.terminate()
return parseInt(result.text)*/
const { data: { text } } = await numberWorker.recognize(image)
return parseInt(text)
}
async function imageToDigit(image) {
const { data: { text } } = await digitWorker.recognize(image)
return parseInt(text)
}
const fingerWidth = 15
const fingerHeight = 15
const stringPositions = [18,43,68,93,118,143]
const fretPositions = [55,80,105,130,155]
const baseFretPositionX = 162
const baseFretPositionY = 51
const baseFretPositionWidth = 24
const baseFretPositionHeight = 24
const stringNames = 'EADGBe'
const jGuitarBaseUrl = 'https://jguitar.com'
async function getChordDiagramBaseFret(inputImg) {
const tmpOutput = await inputImg.clone().extract({ left: baseFretPositionX, top: baseFretPositionY, width: baseFretPositionWidth, height: baseFretPositionHeight }).toBuffer()
const baseFret = await imageToNumber(tmpOutput)
//await inputImg.clone().extract({ left: baseFretPositionX, top: baseFretPositionY, width: baseFretPositionWidth, height: baseFretPositionHeight }).toFile(`extracted - Base Fret: ${baseFret}.png`)
return baseFret
}
async function getChordDiagramFingering(inputImg) {
let tmpOutput
const fingering = []
for ([string,stringPos] of Object.entries(stringPositions)) {
let fingerFound
let fingerFoundFret
for ([relFret,fretPos] of Object.entries(fretPositions)) {
tmpOutput = await inputImg.clone().extract({ left: stringPos, top: fretPos, width: fingerWidth, height: fingerHeight }).toBuffer()
const finger = await imageToDigit(tmpOutput)
//await inputImg.clone().extract({ left: stringPos, top: fretPos, width: fingerWidth, height: fingerHeight }).toFile(`extracted - String: ${string} (${stringNames[string]}) - Relative Fret: ${relFret} - Finger: ${finger}.png`)
if (!isNaN(finger)) {
fingerFound = true
fingering.push(finger)
break
}
}
if (!fingerFound) {
fingering.push(0)
}
}
return fingering
}
const rootNotes = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B']
const bassNotes = rootNotes
const chordSuffixes = [
"Major",
"Minor",
"Diminished",
"Augmented",
"Suspended 2nd",
"Suspended 4th",
"Major Flat 5th",
"Minor Sharp 5th",
"Minor Double Flat 5th",
"Suspended 4th Sharp 5th",
"Suspended 2nd Flat 5th",
"Suspended 2nd Sharp 5th",
"7th",
"Minor 7th",
"Major 7th",
"Minor Major 7th",
"Diminished 7th",
"Augmented 7th",
"Augmented Major 7th",
"7th Flat 5th",
"Major 7th Flat 5th",
"Minor 7th Flat 5th",
"Minor Major 7th Flat 5th",
"Minor Major 7th Double Flat 5th",
"Minor 7th Sharp 5th",
"Minor Major 7th Sharp 5th",
"7th Flat 9th",
"6th",
"Minor 6th",
"6th Flat 5th",
"6th Add 9th",
"Minor 6th Add 9th",
"9th",
"Minor 9th",
"Major 9th",
"Minor Major 9th",
"9th Flat 5th",
"Augmented 9th",
"9th Suspended 4th",
"7th Sharp 9th",
"7th Sharp 9th Flat 5th",
"Augmented Major 9th",
"11th",
"Minor 11th",
"Major 11th",
"Minor Major 11th",
"Major Sharp 11th",
"13th",
"Minor 13th",
"Major 13th",
"Minor Major 13th",
"7th Suspended 2nd",
"Major 7th Suspended 2nd",
"7th Suspended 4th",
"Major 7th Suspended 4th",
"7th Suspended 2nd Sharp 5th",
"7th Suspended 4th Sharp 5th",
"Major 7th Suspended 4th Sharp 5th",
"Suspended 2nd Suspended 4th",
"7th Suspended 2nd Suspended 4th",
"Major 7th Suspended 2nd Suspended 4th",
"5th",
"Major Add 9th"
]
function getChordImgsFromHtml(html) {
const $ = cheerio.load(html)
const chordShapeImgs = []
$('img.img-responsive').each(function() {
chordShapeImgs.push(this)
})
return chordShapeImgs
}
function getPageInfoFromHtml(html) {
const $ = cheerio.load(html)
let [fullMatch, firstChordIndex, lastChordIndex, totalChordCount] = $('body').html().match(/Showing results (\d+) to (\d+) of (\d+) chord/)
firstChordIndex = parseInt(firstChordIndex)
lastChordIndex = parseInt(lastChordIndex)
totalChordCount = parseInt(totalChordCount)
const totalPageCount = Math.ceil(totalChordCount/lastChordIndex)
return {
firstChordIndex,
lastChordIndex,
totalChordCount,
totalPageCount
}
}
async function main() {
await digitWorker.load()
await digitWorker.loadLanguage('eng')
await digitWorker.initialize('eng', 2)
await digitWorker.setParameters({
'tessedit_char_whitelist': '0123456789'
})
await numberWorker.load()
await numberWorker.loadLanguage('eng')
await numberWorker.initialize('eng', 0)
await numberWorker.setParameters({
'tessedit_char_whitelist': '0123456789'
})
/*
const inputImg = await sharp('test.png')
const fingering = await getChordDiagramFingering(inputImg)
const baseFret = await getChordDiagramBaseFret(inputImg)
console.log('Base Fret:', baseFret)
console.log('Fingering detected:', fingering)
process.exit()
*/
for (const rootNote of rootNotes) {
for (const chordSuffix of chordSuffixes) {
for (const bassNote of bassNotes) {
const encRootNote = encodeURIComponent(rootNote)
const encChordSuffix = encodeURIComponent(chordSuffix)
const encBassNote = encodeURIComponent(bassNote)
const url = `${jGuitarBaseUrl}/chord?root=${encRootNote}&chord=${encChordSuffix}&bass=${encBassNote}&labels=finger&gaps=0&fingers=4¬es=sharps`
const html = await rp({ uri: url })
let chordShapeImgs = getChordImgsFromHtml(html)
const { firstChordIndex, lastChordIndex, totalChordCount, totalPageCount } = getPageInfoFromHtml(html)
for (let i=2; i<=totalPageCount; i++) {
let nextPageUrl = `${url}&page=${i}`
const html = await rp({ uri: nextPageUrl })
const newChordShapeImgs = getChordImgsFromHtml(html)
chordShapeImgs = chordShapeImgs.concat(newChordShapeImgs)
}
for (let [i,img] of Object.entries(chordShapeImgs)) {
i = parseInt(i)
let [fullMatch, fingerPositions] = img.attribs.alt.match(/\{([0-9x]+\s[0-9x]+\s[0-9x]+\s[0-9x]+\s[0-9x]+\s[0-9x]+)\}/)
fingerPositions = fingerPositions.split(' ')
const imgUrl = `${jGuitarBaseUrl}${img.attribs.src.replace(/\/\//g,'/')}`
let fullChordName
if (rootNote === bassNote)
fullChordName = `${encRootNote}${encChordSuffix}`
else
fullChordName = `${encRootNote}${encChordSuffix}${encodeURIComponent('/')}${encBassNote}`
const filePath = `./download/diagrams/${fullChordName}.png`
await download(imgUrl, `./download/diagrams/`, {filename: `${fullChordName}.png`})
const inputImg = await sharp(`./download/diagrams/${fullChordName}.png`)
const fingering = await getChordDiagramFingering(inputImg)
console.log('fullChordName', fullChordName)
console.log('fingerPositions', fingerPositions)
console.log('fingering', fingering)
process.exit() // cancel early because we only want to parse one chord diagram for now
}
process.exit()
}
}
}
process.exit()
}
main().catch(console.error)
I also need to add a bit more code to extract the "subsets" for each chord (which unfortunately don't have fingerings) and convert the chord names a bit (e.g. instead of CMajor and CMinor, it should just use C and Cm).
I let the script run and extract the first chord from jguitar completely by itself and it came up with this data:
fullChordName CMajor
fingerPositions [ 'x', '3', '2', '0', '1', '0' ]
fingering [ 0, 3, 2, 0, 1, 0 ]
From
<img src="/images/chordshape/C-Major-C-x%2C3%2C2%2C0%2C1%2C0.png" alt="C chord {x 3 2 0 1 0} chord" class="img-responsive" width="200" height="200"><div class="row">
Wow, cool! Nice and interesting work! Just an idea, but maybe we could use the chords-db as validation set. There wouldn't be a perfect match because chords-db doesn't contain all the chords available on the jGuitar, however we could check manually the differences. Maybe, we could check only the most common occurrence of the chord.
Probably, for simplified suffixes we could use what we have in the chords-db here: https://github.com/tombatossals/chords-db/blob/master/src/db/guitar/suffixes.js
For generating slash chords we could reuse the array of rootNotes ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B'] as base notes, so we would have a three level deep for loop, e.g.: rootNote + suffix + base note => C + m + '/' + A => Cm/A.
Can we parse also barres chords? Could you create a new repository with your code and add me as contributor?
Look closely, I'm already doing the three level deep loop to really get all chords ;)
for (const rootNote of rootNotes) {
for (const chordSuffix of chordSuffixes) {
for (const bassNote of bassNotes) {
The script can indeed also parse barre chords.
Sure, I'll create the repo later today.
All right, many thanks!
@szaza https://github.com/T-vK/jguitar-parser
There are a few TODOs in the file: https://github.com/T-vK/jguitar-parser/blob/master/index.js#L260 https://github.com/T-vK/jguitar-parser/blob/master/index.js#L269
and in addition to that I need to figure out what order the shapes of each chord should be sorted. I guess the subsets should probably be added to the very end as there is no fingering information for those. I also wonder if it would make sense to merge the jguitar chords with the data I extracted from chordsdatabase.com and the data from this repository. Because jguitar may have all the chord shapes, but only has one fingering for each chord and for the subsets it doesn't have any fingerings at all.
The way I'm currently parsing the chord diagrams is not very efficient. All in all it takes about 1 second for each chord diagram. So extracting and parsing all the chords would take about 30 hours. But, I think I'm not gonna bother with making it more efficient.
Hi @T-vK, many thanks for your effort, great work! I think the 1 second runtime/chord is not a problem, because in best case we have to run it only once for the full dataset. Also we can run in parts for smaller datasets to make validations.
That's fantastic job @T-vK. We could parse the chords and make some validations using chords-db and manually as @szaza says.
I'm working too on the gh-pages of react-chords, the idea is accomplish a basic raw visualizator of all the chords that can help us to detect problems with the news imported chords.
Just tell me if you need help with the parsing
It seems like some chords are too crazy to be played on the guitar.
C/F# Minor Major 9th
for instance doesn't have a result on jguitar: https://jguitar.com/chord?root=C&chord=Minor+Major+9th&bass=F%23&labels=finger&gaps=2&fingers=4¬es=sharps
Edit:
But I also have good news. I found a way to get the fingerings for the subsets. :)
If you remove the -true
from the URL, you get the fingerings:
So instead of:
https://jguitar.com/images/chordshape/C-Major-C-x%2C3%2C2%2C0%2Cx%2C0-sharps-finger-true.png
We need:
https://jguitar.com/images/chordshape/C-Major-C-x%2C3%2C2%2C0%2Cx%2C0-sharps-finger.png
Edit2: I'm not so sure anymore if jguitars data is good enough. This one for instance: https://jguitar.com/images/chordshape/C-Major-C-8%2Cx%2C10%2C9%2C8%2C8-sharps-finger.png
The fingering seems very unrealistic. At least I don't think that I could play it this way. I would probably use my thumb on the low E and A instead of my index finger. What do you think?
Edit3: I also found this interesting script: https://jguitar.com/js/chordcalculator.cf36bb7e77075a1e6a4c33d9fee9a608.min.js
It might be worth it to reverse engineer. It seems to be responsible for calculating the subsets and might be able to do even more.
"The fingering seems very unrealistic. At least I don't think that I could play it this way. I would probably use my thumb on the low E and A instead of my index finger. What do you think?" - I also observed that sometimes fingering seems unrealistic.
@T-vK I would like to convert the output of your application according to chords-db format in order to be able to compare with the validation set and later to extend chords-db. Are you still working on the jguitar-parser? If it is ok with you then I'm going to create a new development branch and implement the needed changes there. Please let me know if you are already working on similar feature.
@szaza Feel free to go for it. I just pushed what I got so far. I'm currently running the parser to download all the raw data from jguitar into the cache. I think I'm not gonna add the subsets to the output for now because too many of them seem to be useless.
I already have 40% of the raw jguitar data downloaded and I expect the final output json to be ~10MB in size. But that's with the long chord names. One we have a function to generate the shorter more common chord names it will probably be a little bit smaller.
Ok, then I'm going to implement a converter that converts the output of your application to chords-db format and probably I can solve the chord names problem too. If you think I can parse a part of the jguitar chords with your tool and we could merge the output at the end, however I could do it only tomorrow afternoon. Please let me know if I can help anything else.
I'm going to implement a converter that converts the output of your application to chords-db format and probably I can solve the chord names problem too
That would be great!
If you think I can parse a part of the jguitar chords with your tool
Thanks, but it'll be easier for me when I have all the raw html and png data on one computer. I'll probably have to parse all that data again anyway at some point.
Hi, The implementation of the converter is mainly done. I implemented it as a subproject: https://github.com/T-vK/jguitar-parser/tree/chords-db-converter; Please review it when you have some free time. At the moment it is not able to generate information about barres and capo. In the chords-db there are chords where barres and capo attributes appear, for example:
{
frets: '575688',
fingers: '131244',
barres: [5, 8],
capo: true
},
I cannot find any information regarding to the barres and capo in the output of the jguitar-parser tool. Is that information missing from the generated chords.json?
Please review it when you have some free time.
Awesome, I will!
Is that information missing from the generated chords.json?
I don't really see the value of the barre/capo information. But I would say if the same finger has to be positioned on multiple strings on the same fret, then you have a barre. And if the index finger makes a barre, then you could alternatively use a capo instead of the index finger and play it as an open chord. In that case, however, the fingering would most likely be very different. But I guess that's when you would want to set the capo
property to true.
I'm almost done scraping the data btw. It crashed a couple of times, but I'm at 95% now.
Hi, I have finished a basic chord visualization tool that can help us to review the new chords imported from a parser: https://tombatossals.github.io/react-chords
Maybe some capo/barre info can be added to the imported set manually. The test could detect basic typos, and we can review the import and improve the queality of the chord in the JSON with more properties.
Great job guys!
Okay, it's done. The final json file ist about 13.5MB in size. I also created a smaller version that only contains one shape per chord which is about 1.2MB in size. I've uploaded the files into this repo: https://github.com/T-vK/chord-collection
Now we need to verify that the data really is good enough.
That's great @T-vK. I'm going to work in an import tool based on your 13.5MB JSON which will merge the new chords with the actual chords-db structure. My idea is to mantain the already existing chords in the DB, and add the new chords that does not exists.
After that, some basic testing and some visually review of them and the job will be done.
What do you think guys?