Contributing to Pysteps

Welcome! pySTEPS is a community-driven initiative for developing and maintaining an easy to use, modular, free and open source Python framework for short-term ensemble prediction systems.

If you haven’t already, take a look at the project’s README.rst file and the pysteps documentation.

There are many ways to contribute to pysteps:

  • contributing bug reports and feature requests
  • contributing documentation
  • code contributions new features or bug fixes
  • contribute with usage examples

Our main forum for discussion is the project’s GitHub issue tracker. This is the right place to start a discussion, report a bug, or request a new feature.

Workflow for code contributions

We welcome all kind of contributions, from documentation updates, a bug fix, or a new feature. If your new feature will take a lot of work, we recommend creating an issue with the enhancement tag to encourage discussions.

We use the usual GitHub pull-request flow, which may be familiar to you if you’ve contributed to other projects on GitHub.

First Time Contributors

If you are interested in helping to improve pysteps, the best way to get started is by looking for “Good First Issue” in the issue tracker.

In a nutshell, the main steps to follow for contributing to pysteps are:

  • Setup the development environment
  • Fork the repository
  • Create a new branch for each contribution
  • Read the Code Style guide
  • Work on your changes
  • Test your changes
  • Push to your fork repository and create a new PR in GitHub.

Setup the Development environment

The recommended way to setup up the developer environment is the Anaconda (commonly referred as Conda). Conda quickly installs, runs and updates packages and their dependencies. It also allows to easily create, save, load and switch between different environments on your local computer.

The developer environment can be created using the environment_dev.yml file in the project’s root directory running the command:

conda env create -f environment_dev.yml

This will create the pysteps_dev environment that can be activated using:

conda activate pysteps_dev

Once the environment is created, the package can be installed in development mode, in such a way that the project appears to be installed, but yet is still editable from the source tree. See instructions in the Installing pysteps section.

Fork the repository

Once you have set the development environment, the next step is creating your local copy of the repository where you will commit your modifications. The steps to follow are:

  1. Set up Git in your computer.

  2. Create a GitHub account (if you don’t have one).

  3. Fork the repository in your GitHub.

  4. Clone local copy of your fork. For example:

    git clone https://github.com/<your-account>/pysteps.git

Done!, now you have a local copy of pysteps git repository. If you are new to GitHub, below you can find a list of useful tutorials:

Create a new branch

As a collaborator, all the new contributions that you want should be done in a new branch under your forked repository. Working on the master branch is reserved for Core Contributors only. Core Contributors are developers that actively work and maintain the repository. They are the only ones who accept pull requests and push commits directly to the pysteps repository.

For more information on how to create and work with branches, see “Branches in a Nutshell” in the Git documentation

Code Style

Although it is not strictly enforced yet, we strongly suggest to follow the PEP8 coding standards. Two popular modules used to check pep8 compliance are pycodestyle and pylint that can be installed using pip:

pip install pylint
pip install pycodestyle

or using anaconda:

conda install pylint
conda install pycodestyle

For further information instructions, the reader is referred to their official documentation.

Coding style summary

For quick reference, these are the most important good coding practices to follow:

  • Always use 4 spaces for indentation (don’t use tabs).

  • Write UTF-8 (add # -*- coding: utf-8 -*- at the top of each file).

  • Max line-length: 79 characters.

  • Always indent wrapped code for readability.

  • Avoid extraneous whitespace.

  • Don’t use whitespace to line up assignment operators (=, :).

  • Spaces around = for assignment.

  • No spaces around = for default parameter values (keywords).

  • Spaces around mathematical operators, but group them sensibly.

  • No multiple statements on the same line.

  • Naming conventions:

    Function names, variable names, and filenames should be descriptive and self explanatory. Avoid using abbreviations that are ambiguous or unfamiliar to readers outside your project, and do not abbreviate by deleting letters within a word. Avoid single letter variables if possible and use more verbose names for clarity. An exception for this are indexes in loops (i, j, k, etc).

    The following table summarizes the conventions:

    Type Public Internal
    Packages lower_with_under
    Modules lower_with_under _lower_with_under
    Classes CapWords _CapWords
    Exceptions CapWords
    Functions lower_with_under() _lower_with_under()
    Global/Class Constants CAPS_WITH_UNDER _CAPS_WITH_UNDER
    Global/Class Variables lower_with_under _lower_with_under
    Instance Variables lower_with_under _lower_with_under (protected)
    Method Names lower_with_under() _lower_with_under() (protected)
    Function/Method Parameters lower_with_under
    Local Variables lower_with_under

    Source: Google’s python style guide

  • Create an ignored variable:

    If you need to assign something (for instance, in Unpacking) but will not need that variable, use __ (double underscore):

    precip, __, metadata = import_bom_rf3('example_file.bom')

    Many Python style guides recommend the use of a single underscore “_” rather than the double underscore “__” recommended here. The issue is that “_” is commonly used as an alias for the gettext() function, and is also used at the interactive prompt to hold the value of the last operation. Using a double underscore instead is just as clear and eliminates the risk of accidentally interfering with either of these other use cases. (Source: https://docs.python-guide.org/writing/style/)

  • Zen of Python (PEP 20), the guiding principles for Python’s design:

    >>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

For a detailed description of a pythonic code style check these guidelines:

Auto-formatters

Formatting code to PEP8 style is a time consuming process. Instead of manually formatting code before a commit to to PEP8 style, you can use auto-format packages which automatically formats Python code to conform to the PEP 8 style guide.

If your development environment does not include auto-formatting capabilities, we recommend using black, which can be installed by any of the following options:

conda install black

#For the latest version:
conda install -c conda-forge black

pip install black

Check the official documentation for more information.

Docstrings

Every module, function, or class must have a docstring that describe its purpose and how to use it, following the conventions described in the PEP 257 and the Numpy’s docstrings format.

Here is a summary of the most important rules:

  • One-line docstrings Triple quotes are used even though the string fits on one line. This makes it easy to later expand it.
  • A one-line docstring is a phrase ending in a period.
  • All docstrings should be written in imperative (“”“Return some value.”“”) mood rather than descriptive mood (“”“Returns some value.”“”).

Here is an example of a docstring:

def adjust_lag2_corrcoef1(gamma_1, gamma_2):
    """A simple adjustment of lag-2 temporal autocorrelation coefficient to
    ensure that the resulting AR(2) process is stationary when the parameters
    are estimated from the Yule-Walker equations.

    Parameters
    ----------

    gamma_1 : float
      Lag-1 temporal autocorrelation coeffient.

    gamma_2 : float
      Lag-2 temporal autocorrelation coeffient.

    Returns
    -------

    out : float
      The adjusted lag-2 correlation coefficient.
    """

Working on changes

IMPORTANT

If your changes will take a significant amount of work, we highly recommend opening an issue first, explaining what do you want to do and why. It is better to start the discussions early in case that other contributors disagree with what you would like to do or have ideas that will help you do it.

Collaborators guidelines

As a collaborator, all the new contributions that you want should be done in a new branch under your forked repository. Working on the master branch is reserved for Core Contributors only. Core Contributors are developers that actively work and maintain the repository. They are the only ones who accept pull requests and push commits directly to the pysteps repository.

To include the contributions for collaborators, we use the usual GitHub pull-request flow. In their simplest form, pull requests are a mechanism for a collaborator to notify to the pysteps project about a completed feature”.

Once your proposed changes are ready, you need to create a pull request via your GitHub account. Afterward, the core developers review the code and merge it into the master branch. Be aware that pull requests are more than just a notification, they are also an excellent place for discussing the proposed feature. If there is any problem with the changes, the other project collaborators can post feedback and the author of the commit can even fix the problems by pushing follow-up commits to feature branch.

Do not squash your commits after you have submitted a pull request, as this erases context during the review. The commits will be squashed when the pull request is merged.

To keep you forked repository clean, we suggest deleting branches for once the Pull Requests (PRs) are accepted and merged.

Once you’ve created a pull request, you can push commits from your topic branch to add them to your existing pull request. These commits will appear in chronological order within your pull request and the changes will be visible in the “Files changed” tab.

Other contributors can review your proposed changes, add review comments, contribute to the pull request discussion, and even add commits to the pull request.

Important: each PR should should only address a single objective (e.g. fix a bug, improve documentation, etc). Pushing changes to an open PR that are outside its objective are highly discouraged. Under this circumstances, the recommended way to proceed is creating a new PR for changes, clearly explaining their goal.

Testing your changes

Before committing changes or creating pull requests, check that the all the tests in the pysteps suite pass. See the Testing pySTEPS for the instruction to run the tests.

Although it is not strictly needed, we suggest creating minimal tests for new contributions to ensure that it achieves the desired behavior. Pysteps uses the pytest framework, that it is easy to use and also supports complex functional testing for applications and libraries. Check the pytests official documentation for more information.

The tests should be placed under the pysteps.tests module. The file should follow the test_*.py naming convention and have a descriptive name.

A quick way to get familiar with the pytest syntax and the testing procedures is checking the python scripts present in the pysteps test module.

Core developer guidelines

Working directly on the master branch is discouraged and is reserved only for small changes and updates that do not compromise the stability of the code. The master branch is a production branch that is ready to be deployed (cloned, installed, and ready to use). In consequence, this master branch is meant to be stable.

The pysteps repository uses a Travis CI, a Continuous Integration service that automatically runs a series of tests every time you commit to GitHub. In that way, your modifications along with the entire package is tested.

Pushing untested or work-in-progress changes to the master branch can potentially introduce bugs or brake the stability of the package. Since the tests takes around 10 minutes and the are run after the commit was pushed, any errors introduced in that commit will be noticed after the stable in the master branch was compromised. In addition, other developers start working on a new feature from master, they may start a potentially broken state.

Instead, it is recommended to work on each new feature in its own branch, which can be pushed to the central repository for backup/collaboration. When you’re done with the development work on the feature, then you can merge the feature branch into the master or submit a Pull Request. This approach has two main advantages:

  • Every commit on the feature branch is tested using Travis CI. If the tests fail, they do not affect the master branch.
  • Once the new feature, improvement, or bug correction is finished and the all tests passed, the commits history can be squashed into a single commit and then merged into the master branch.

This helps approach helps to keep the commits history clean and allows experimentation in the branch without compromising the stability of the package.

Processing pull requests

Core developers should follow these rules when processing pull requests:

  • Always wait for tests to pass before merging PRs.

  • Use “Squash and merge” to merge PRs.

  • Delete branches for merged PRs (by core devs pushing to the main repo).

  • Edit the final commit message before merging to conform to the following style to help having a clean git log output:

    • When merging a multi-commit PR make sure that the commit message doesn’t contain the local history from the committer and the review history from the PR. Edit the message to only describe the end state of the PR.
    • Make sure there is a single newline at the end of the commit message. This way there is a single empty line between commits in git log output.
    • Split lines as needed so that the maximum line length of the commit message is under 80 characters, including the subject line.
    • Capitalize the subject and each paragraph.
    • Make sure that the subject of the commit message has no trailing dot.
    • Use the imperative mood in the subject line (e.g. “Fix typo in README”).
    • If the PR fixes an issue, make sure something like “Fixes #xxx.” occurs in the body of the message (not in the subject).

Preparing a new release

Core developers should follow the steps to prepare a new release (version):

  1. Before creating the actual release in GitHub, be sure that every item in the following checklist was followed:

    • In the file setup.py, update the version=”X.X.X” keyword in the setup function.
    • Update the version in PKG-INFO file.
    • If new dependencies were added to pysteps since the last release, add them to the environment.yml, requirements.txt, and requirements_dev.txt files.

  2. Create a new release in GitHub following these guidelines. Include a detailed changelog in the release.

  3. Generating the source distribution for new pysteps version and upload it to the Python Package Index (PyPI). See Packaging the pysteps project for a detailed description of this process.

  4. Update the conda-forge pysteps-feedstock following this guidelines: Updating the conda-forge pysteps-feedstock

Credits

This documents was based in contributors guides of two Python open source projects: