Skip to content

feat: Update project build and configuration #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pbadillatorrealba merged 5 commits into from
Jul 29, 2025
Merged

feat: Update project build and configuration #65

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
34b6210
chore: centralize all build and package config in pyproject.toml
pbadillatorrealba Jul 29, 2025
48141d5
chore: update sphinx build conf
pbadillatorrealba Jul 29, 2025
9f87cd6
chore: update documentation and changelog
pbadillatorrealba Jul 29, 2025
1783d0e
chore: update annotations
pbadillatorrealba Jul 29, 2025
caedb71
chore: update ci dependencies installation, improve mypy configuratio…
pbadillatorrealba Jul 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .flake8
View file
Open in desktop

This file was deleted.

4 changes: 1 addition & 3 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
if [ -f requirements-dev.txt ]; then pip install -r requirements-dev.txt; fi
pip install -e ".[dev]"
- name: Test with pytest
run: |
pytest tests
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ repos:
- id: trailing-whitespace
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.12.3
rev: v0.12.5
hooks:
# Run the linter.
- id: ruff-check
Expand Down
7 changes: 3 additions & 4 deletions .readthedocs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,14 @@ formats:
build:
os: ubuntu-22.04
tools:
python: "3.11"

python: "3.12"

sphinx:
configuration: docs/conf.py

python:
install:
- requirements: requirements.txt
- requirements: requirements-dev.txt
- method: pip
path: .
extra_requirements:
- dev
2 changes: 1 addition & 1 deletion LICENSE
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2022 WEFE Team
Copyright (c) 2025 WEFE Team

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
2 changes: 0 additions & 2 deletions MANIFEST.in
View file
Open in desktop

This file was deleted.

132 changes: 98 additions & 34 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,91 +55,155 @@ The official documentation can be found at this `link <https://wefe.readthedocs.
Installation
============

There are two different ways to install WEFE:
WEFE requires Python 3.10 or higher. There are two different ways to install WEFE:


To install the package with ``pip`` ::
**Install with pip** (recommended)::

pip install wefe

- With conda:

To install the package with ``conda``::
**Install with conda**::

conda install -c pbadilla wefe

**Install development version**::

pip install git+https://github.com/dccuchile/wefe.git

**Install with development dependencies**::

pip install "wefe[dev]"

**Install with PyTorch support**::

pip install "wefe[pytorch]"


Requirements
------------

These package will be installed along with the package, in case these have not already been installed:
WEFE automatically installs the following dependencies:

1. numpy
2. scipy
3. scikit-learn
4. scipy
5. pandas
6. gensim
7. plotly
8. requests
9. tqdm
10. semantic_version
- gensim (>=3.8.3)
- numpy (<=1.26.4)
- pandas (>=2.0.0)
- plotly (>=6.0.0)
- requests (>=2.22.0)
- scikit-learn (>=1.5.0)
- scipy (<1.13)
- semantic_version (>=2.8.0)
- tqdm (>=4.0.0)

Contributing
------------

You can download the code executing ::
To contribute to WEFE development:

1. **Clone the repository**::

git clone https://github.com/dccuchile/wefe
cd wefe

2. **Install in development mode with all dependencies**::

pip install -e ".[dev]"

3. **Run tests to ensure everything works**::

pytest tests

To contribute, visit the `Contributing <https://wefe.readthedocs.io/en/latest/user_guide/contribute.html>`_ section in the documentation.
4. **Make your changes and run tests again**

5. **Follow our coding standards**:
- Use ``ruff`` for code formatting: ``ruff format .``
- Check code quality: ``ruff check .``
- Run type checking: ``mypy wefe``

For detailed contributing guidelines, visit the `Contributing <https://wefe.readthedocs.io/en/latest/user_guide/contribute.html>`_ section in the documentation.

Development Requirements
------------------------

To install the necessary dependencies for the development, testing and compilation
of WEFE documentation, run ::
To install WEFE with all development dependencies for testing, documentation building, and code quality tools::

pip install "wefe[dev]"

pip install -r requirements-dev.txt
This installs additional packages including:

- pytest and pytest-cov for testing
- sphinx and related packages for documentation
- ruff for code formatting and linting
- mypy for type checking
- ipython for interactive development


Testing
-------

All unit tests are in the wefe/tests folder. It uses ``pytest`` as a framework to
run them.
All unit tests are in the ``tests/`` folder. WEFE uses ``pytest`` as the testing framework.

To run the test, execute::
To run all tests::

pytest tests

To check the coverage, run::
To run tests with coverage reporting::

pytest tests --cov=wefe --cov-report=html

pytest tests --cov-report xml:cov.xml --cov wefe
To run a specific test file::

And then::
pytest tests/test_datasets.py

coverage report -m
Coverage reports will be generated in ``htmlcov/`` directory.


Build the documentation
-----------------------

The documentation is created using sphinx.
It can be found in the docs folder at the root of the project.
To compile the documentation, run:
The documentation is built using Sphinx and can be found in the ``docs/`` folder.

To build the documentation::

.. code-block:: bash
cd docs
make html

Or using the development environment::

pip install "wefe[dev]"
cd docs
make html

Then, you can visit the documentation at ``docs/_build/html/index.html``
The built documentation will be available at ``docs/_build/html/index.html``

Changelog
=========

Version 1.0.0
-------------------

**Major Release - Breaking Changes**

- **Python 3.10+ Required**: Dropped support for Python 3.6-3.9
- **Modern Packaging**: Migrated from ``setup.py`` to ``pyproject.toml``
- **Updated Dependencies**: All packages updated for modern Python ecosystem

**New Features**:
- Robust dataset fetching with retry mechanism and exponential backoff
- HTTP 429 (rate limiting) and timeout error handling
- Optional dependencies: ``pip install "wefe[dev]"`` and ``"wefe[pytorch]"``
- Dynamic version loading from ``wefe.__version__``

**Core Improvements**:
- **WordEmbeddingModel**: Enhanced type safety, better gensim compatibility, improved error handling
- **BaseMetric**: Refactored input validation, standardized ``run_query`` methods across all metrics
- **Testing**: Converted to pytest patterns with monkeypatch, comprehensive test coverage
- **Code Quality**: Migration from flake8 to Ruff, enhanced documentation with detailed docstrings

**Development Workflow**:
- GitHub Actions upgraded with Python 3.10-3.13 matrix testing
- Pre-commit hooks enhanced with JSON/TOML validation and security checks
- Modernized Sphinx documentation configuration
- Updated benchmark documentation and metrics comparison tables

Version 0.4.1
-------------------

Expand Down
61 changes: 37 additions & 24 deletions docs/benchmark/benchmark.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1096,29 +1096,42 @@ methods they implement to date.
Fairness Metrics
~~~~~~~~~~~~~~~~

=================== ========================= ==== === =========== ===================
Metric Implementable in WEFE WEFE FEE Responsibly EmbeddingBiasScores
=================== ========================= ==== === =========== ===================
WEAT ✔ ✔ ✔ ✔ ✔
WEAT ES ✔ ✔ ✖ ✖ ✖
RNSB ✔ ✔ ✖ ✖ ✖
RIPA ✔ ✔ ✖ ✖ ✔
ECT ✔ ✔ ✖ ✖ ✖
RND ✔ ✔ ✖ ✖ ✖
MAC ✔ ✔ ✖ ✖ ✔
Direct Bias ✔ ✖ ✔ ✔ ✔
SAME ✔ ✖ ✖ ✖ ✔
Generalized WEAT ✔ ✖ ✖ ✖ ✔
IndirectBias ✖ ✖ ✖ ✔ ✖
GIPE ✖ ✖ ✔ ✖ ✖
PMN ✖ ✖ ✔ ✖ ✖
Proximity Bias ✖ ✖ ✔ ✖ ✖
=================== ========================= ==== === =========== ===================

The following metrics are not compatible with WEFE because they do not align with the abstractions defined by the framework:

- IndirectBias, which accepts only two words and a gender direction as input, relying on a previously calculated bias direction.
- GIPE, PMN, and Proximity Bias, which evaluate word embedding models before and after debiasing using auxiliary mitigation methods.
+---------------------+----------------------------+------+------+-------------+-----------------------+
| Metric | Implementable in WEFE | WEFE | FEE | Responsibly | EmbeddingBiasScores |
+=====================+============================+======+======+=============+=======================+
| WEAT | ✔ | ✔ | ✔ | ✔ | ✔ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| WEAT ES | ✔ | ✔ | ✖ | ✖ | ✖ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| RNSB | ✔ | ✔ | ✖ | ✖ | ✖ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| RIPA | ✔ | ✔ | ✖ | ✖ | ✔ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| ECT | ✔ | ✔ | ✖ | ✖ | ✖ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| RND | ✔ | ✔ | ✖ | ✖ | ✖ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| MAC | ✔ | ✔ | ✖ | ✖ | ✔ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| Direct Bias | ✔ | ✖ | ✔ | ✔ | ✔ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| SAME | ✔ | ✖ | ✖ | ✖ | ✔ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| Generalized WEAT | ✔ | ✖ | ✖ | ✖ | ✔ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| IndirectBias | ✖ | ✖ | ✖ | ✔ | ✖ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| GIPE | ✖ | ✖ | ✔ | ✖ | ✖ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| PMN | ✖ | ✖ | ✔ | ✖ | ✖ |
+---------------------+----------------------------+------+------+-------------+-----------------------+
| Proximity Bias | ✖ | ✖ | ✔ | ✖ | ✖ |
+---------------------+----------------------------+------+------+-------------+-----------------------+

Metrics marked as "✔" in the "Implementable in WEFE" column can be implemented directly within
the WEFE framework using word sets as input.
Metrics marked as "✖" require additional representations such as gender directions
or apply before/after transformations, and are therefore currently out of WEFE's scope.


Mitigation algorithms
Expand All @@ -1143,7 +1156,7 @@ libraries analyzed in this benchmark study.
==================================================== ========================================= ========================================================== ========================================== ====================================
Item WEFE FEE Responsibly EmbeddingBiasScores
==================================================== ========================================= ========================================================== ========================================== ====================================
Implemented Metrics 7 7 3 6
Implemented Metrics 7 5 3 6
Implemented Mitigation Algorithms 5 3 1 0
Extensible Easy Easy Difficult, not very modular. Easy
Well-defined interface for metrics ✔ ✖ ✖ ✔
Expand Down
Loading