Skip to content

Commit

Permalink
docs: refactor and update readme [no-ci]
Browse files Browse the repository at this point in the history
  • Loading branch information
k4black committed Jul 12, 2023
1 parent a784b3f commit c0757c0
Showing 1 changed file with 28 additions and 16 deletions.
44 changes: 28 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
[![PyPI version](https://badge.fury.io/py/codebleu.svg)](https://badge.fury.io/py/codebleu)


Unofficial `CodeBLEU` implementation with Linux and MacOS supports available with PyPI and HF HUB.
This repository contains an unofficial `CodeBLEU` implementation that supports Linux and MacOS. It is available through `PyPI` and the `evaluate` library.

Based on original [CodeXGLUE/CodeBLEU](https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-to-code-trans/evaluator/CodeBLEU) code -- refactored, build for macos, tested and fixed multiple crutches to make it more usable.
The code is based on the original [CodeXGLUE/CodeBLEU](https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-to-code-trans/evaluator/CodeBLEU) and updated version by [XLCoST/CodeBLEU](https://github.com/reddy-lab-code-research/XLCoST/tree/main/code/translation/evaluator/CodeBLEU). It has been refactored, tested, built for macOS, and multiple improvements have been made to enhance usability

Keywords for `C`, `C#`, `C++`, `JavaScript`, and `PHP` copied from updated version by [XLCoST/CodeBLEU](https://github.com/reddy-lab-code-research/XLCoST/tree/main/code/translation/evaluator/CodeBLEU).
Available for: `Python`, `C`, `C#`, `C++`, `Java`, `JavaScript`, `PHP`.

---

Expand All @@ -25,6 +25,21 @@ In a nutshell, `CodeBLEU` is a weighted combination of `n-gram match (BLEU)`, `w
The metric has shown higher correlation with human evaluation than `BLEU` and `accuracy` metrics.


## Installation

As this library require `so` file compilation it is platform dependent.
Currently available for `Linux` (manylinux) and `MacOS` with Python 3.8+.

The metrics is available as [pip package](https://pypi.org/project/codebleu/) and can be installed as indicated above:
```bash
pip install codebleu
```
or directly from git repo:
```bash
pip install git+https://github.com/k4black/codebleu.git
```


## Usage

```python
Expand All @@ -43,7 +58,7 @@ print(result)
# 'dataflow_match_score': 1.0
# }
```
where calc_codebleu takes the following arguments:
where `calc_codebleu` takes the following arguments:
- `refarences` (`list[str]` or `list[list[str]]`): reference code
- `predictions` (`list[str]`) predicted code
- `lang` (`str`): code language, see `codebleu.AVAILABLE_LANGS` for available languages (python, c_sharp c, cpp, javascript, java, php at the moment)
Expand All @@ -57,7 +72,7 @@ and outputs the `dict[str, float]` with following fields:
- `syntax_match_score`: `syntax_match` score (AST match)
- `dataflow_match_score`: `dataflow_match` score

Alternatively, you can use `k4black/codebleu` from HuggingFace Spaces:
Alternatively, you can use `k4black/codebleu` from HuggingFace Spaces (`codebleu` package required):
```python
import evaluate
metric = evaluate.load("dvitel/codebleu")
Expand All @@ -70,20 +85,17 @@ result = metric.compute([reference], [prediction], lang="python", weights=(0.25,

Feel free to check the HF Space with online example: [k4black/codebleu](https://huggingface.co/spaces/k4black/codebleu)

## Installation

Requires Python 3.8+
## Contributing

The metrics can be installed with pip and used as indicated above:
```bash
pip install codebleu
```
Contributions are welcome!
If you have any questions, suggestions, or bug reports, please open an issue on GitHub.


## License

This project is licensed under the terms of the MIT license.

alternatively the metric is available as [k4black/codebleu](https://huggingface.co/spaces/k4black/codebleu) in `evaluate` (lib installation required):
```python
import evaluate
metric = evaluate.load("k4black/codebleu")
```

## Citation

Expand Down

0 comments on commit c0757c0

Please sign in to comment.