A Collaborative Repository of Natural Language Transformations

NL-Augmenter

The NL-Augmenter is a collaborative effort intended to add transformations of datasets dealing with natural language. Transformations augment text datasets in diverse ways, including: introducing spelling errors, translating to a different language, randomizing names and numbers, paraphrasing … and whatever creative augmentation you contribute to the benchmark. We invite submissions of transformations to this framework by way of GitHub pull request, through September 1, 2021. All submitters of accepted transformations (and filters) will be included as co-authors on a paper announcing this framework.

Submission timeline

A transformation can be revised between the pull request submission and pull request merge deadlines. We will provide reviewer feedback to help with the revisions.

The transformations which are already accepted to NL-Augmenter are summarized in this table. Transformations undergoing review can be seen as pull requests.

Colab notebook

![Open In Colab”/> To quickly see transformations and filters in action, run through our colab notebook.

Installation

Requirements

• Python 3.7

Instructions

# When creating a new transformation, replace this with your forked repository (see below)
git clone https://github.com/GEM-benchmark/NL-Augmenter.git
cd NL-Augmenter
python setup.py sdist
pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz

How do I create a transformation?

Setup

First, fork the repository in GitHub! :fork_and_knife:

<img src="https://pythonawesome.com/content/images/2021/06/68747470733a2f2f646f63732e6769746875622e636f6d2f6173736574732f696d616765732f68656c702f7265706f7369746f72792f666f726b5f627574746f6e2e6a7067.jpg” alt="68747470733a2f2f646f63732e6769746875622e636f6d2f6173736574732f696d616765732f68656c702f7265706f7369746f72792f666f726b5f627574746f6e2e6a7067](https://colab.research.google.com/assets/colab-badge.svg)

Your fork will have its own location, which we will call PATH_TO_YOUR_FORK.

Next, clone the forked repository and create a branch for your transformation, which here we will call my_awesome_transformation:

git clone $PATH_TO_YOUR_FORK
cd NL-Augmenter
git checkout -b my_awesome_transformation

We will base our transformation on an existing example.

Create a new transformation directory by copying over an existing transformation:

cd transformations/
cp -r butter_fingers_perturbation my_awesome_transformation
cd my_awesome_transformation

Creating a transformation

* In the file `transformation.py`, rename the class `ButterFingersPerturbation` to `MyAwesomeTransformation` and choose one of the interfaces from the `interfaces/` folder. See the full list of options here.

• Now put all your creativity in implementing the generate method. If you intend to use external libraries, add them with their version numbers in requirements.txt

• Update my_awesome_transformation/README.md to describe your transformation.

Testing and evaluating (Optional)

Once you are done, add at least 5 example pairs as test cases in the file test.json so that no one breaks your code inadvertently.

Once the transformation is ready, test it:

pytest -s --t=my_awesome_transformation

If you would like to evaluate your transformation against a common 🤗HuggingFace model, we encourage you to check evaluation

Code Styling To standardized the code we use the black code formatter which will run at the time of pre-commit.

To use the pre-commit hook, install pre-commit with pip install pre-commit (should already be installed if you followed the above instructions).

Then run pre-commit install to install the hook. On future commits, you should see the black code formatter is run on all python files you’ve staged for commit.

Submitting

Once the tests pass and you are happy with the transformation, submit them for review.

First, commit and push your changes:

git add transformations/my_awesome_transformation/*
git commit -m "Added my_awesome_transformation"
git push --set-upstream origin my_awesome_transformation

Finally, submit a pull request.

The last git push command prints a URL that can be copied into a browser to initiate such a pull request.

Alternatively, you can do so from the GitHub website.

:sparkles: Congratulations, you’ve submitted a transformation to NL-Augmenter! :sparkles:

GitHub

https://github.com/GEM-benchmark/NL-Augmenter

Source: https://pythonawesome.com/a-collaborative-repository-of-natural-language-transformations/