codebleu
codebleu copied to clipboard
k4black
Pip compatible CodeBLEU metric implementation available for linux/macos/win
Hi @k4black , I have the following error when performing evaluation using CodeBleu in my HuggingFace training loop. Do you know what the issue could be? ``` [cut from longer...
As https://github.com/microsoft/CodeXGLUE/issues/152 I found the same problem for data-flow score calculation as addressed on the above in line 121 and 122 https://github.com/k4black/codebleu/blob/main/codebleu/dataflow_match.py#L121C40-L121C44 There are two possible way to solve it:...
Hi, I am comparing the code lines: codeline1= 'a = !((f >> 4) & 0x01);' codeline2= 'a=!(((f >> 4) & 1U)!=0?true:false;)' the metric is CodeBLEU score: {'codebleu': 0.4474481492943273, 'ngram_match_score': 0.21711852081087685,...
Hello @k4black , How might I be able to add support for Fortran? Thanks
Expected behaviour for these two metrics would be that the weighted ngram match score with a weight of 1 for all ngrams would be equivalent to the unweighted ngram match...
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.21.1 to 2.21.2. Release notes Sourced from pypa/cibuildwheel's releases. v2.21.2 ✨ Adds support for building 32-bit armv7l wheels on musllinux. On a Linux system with emulation set...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
I failed on the given example with a typeerror. --------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[1], [line 6](vscode-notebook-cell:?execution_count=1&line=6) [3](vscode-notebook-cell:?execution_count=1&line=3) prediction = "def add ( a , b )...
Hi, Thank you for sharing your CodeBLEU package. Could you please explain why is there an `or 1` expression in the condition below in your implementation? ` + theta *...
I noticed when calculating CodeBLEU on a snippet without language keywords that the results returned by the BLEU and weighted n-gram match components are not equal to each other. Looking...
