A Reinforcement Learning approach to User Group Behavior Imitation

Python code for the paper "A Reinforcement Learning approach to User Group Behavior Imitation". The paper was accepted and published at the ECIR 2026 Synirgy Workshop.

Cloning and pulling

This git repo also provides GloVe as a submodule. When cloning it you can do git clone --recursive https://github.com/strg-at/paper-rl-semantic-trajectories-python.git to automatically also download the submodule.

Pulling changes for the submodules should always be done with:

git submodule update --remote --rebase

Running

Installing

This Python project utilizes modern packaging standards defined in the pyproject.toml configuration file. Project management tasks like environment setup and dependency installation are handled using UV. The pyproject.toml configuration file can be and is also used to store 3rd party tools configurations, such as ruff, basedpyright etc.

Using UV

UV is a command-line tool that needs to be installed first. You can typically install it using pip, pipx, or your system's package manager if available.

Refer to the official UV installation guide for the most up-to-date methods. A common way is:

# Using pip (ensure pip is up-to-date)
pip install uv

# Or using pipx (recommended for CLI tools)
pipx install uv

You can then install the project dependencies in a virtual environment with:

uv sync --extra cpu  # use --extra cuda if you're running on nvidia or --extra metal if running on Mac Metal cpus

If you also want development tools and libraries:

uv sync --extra cpu --group dev

You can then enable the environment with:

source .venv/bin/activate

If you want to run the experiments, you should also install the clustering group with:

uv sync --group clustering --extra cuda

This will install faiss-cpu. If you're running on GPU, you might want to compile FAISS from source. See FAISS' Github instructions.

Notice: UV can also manage python versions. See Install Python.

Dependency management

You can add a dependency with:

uv add package-name

If the dependency should only be used in some cases (e.g., dev dependencies), then use:

uv add --group dev package-name

Building GloVe

Run:

make all

to compile both our glove_ffi extension, and glove original code

Dataset

The original .csv files of the dataset we use can be downloaded from https://www.kaggle.com/datasets/mkechinov/ecommerce-behavior-data-from-multi-category-store

Notice that there is more than one file: the whole dataset goes from October data to April data.

Generating GloVe embeddings

In order to generate GloVe embeddings you need to have:

1. a .txt file containing the training dataset (glove_trajectories.txt)
2. a vocabulary .txt file (vocab.txt);
3. a cooccurrence binary file (cooccurrence.shuf.bin)

We will generate the first file with the bin/glove_preprocessing.py script. The other two files can be generated with GloVe original C code (which you should have in the submodules directory, see Cloning and pulling).

Once you have downloaded the dataset, use the following python code to merge the csv files and generate a single .parquet file:

import duckdb

duckdb.sql("COPY (SELECT * FROM read_csv(['2019-Oct.csv.gz', '2019-Nov.csv.gz', ...])) TO 'alldata.parquet'")

You should then run the bin/create_graph.py script which will generate several files. Look at the top of the script for configuration options (via env variables).

python bin/create_graph.py

Then, you can generate the glove_trajectories.txt file with:

python bin/glove_preprocessing.py --path trajectories.parquet --output-file glove_trajectories.txt

Feel free to adjust the parameters to your liking.

Now, we can generate the vocabulary and the cooccurence.shuf.bin file. Before running this, check the demo.sh file and see if you want to change any parameter:

cd submodules/GloVe && ./demo.sh

Running on CPU

If you cannot or do not want to run on GPU, add the following line to demo.sh:

$BUILDDIR/glove -save-file $SAVE_FILE -threads $NUM_THREADS -input-file $COOCCURRENCE_SHUF_FILE -x-max $X_MAX -iter $MAX_ITER -vector-size $VECTOR_SIZE -binary $BINARY -vocab-file $VOCAB_FILE -verbose $VERBOSE

This will compute the embeddings using Glove C code, and will output a vectors.txt file.

Running on GPU

Warning

We currently do not recommend to run GloVe on GPU, as this code is not well tested and optimized. The pure C original implementation is fast enough, and even faster, in most cases.

For this, you will need the ./experiments/train_glove_embeddings.py script. Run it with:

python ./experiments/train_glove_embeddings.py --vocab-file vocab.txt --cooccurr-file cooccurence.shuf.bin --embeddings-size 100 --embs-save-path glove_vectors.pt --glove-vectors-save-path glove_vectors.txt

Notice that the script also exposes a tinygrad model, in case you have an AMD or Intel GPU. You can run it with:

python ./experiments/train_glove_embeddings.py --vocab-file vocab.txt --cooccurr-file cooccurence.shuf.bin --embeddings-size 100 --embs-save-path glove_vectors.pt --glove-vectors-save-path glove_vectors.txt --use-tinygrad

Using GloVe embeddings

The generated embeddings will be in a file called vectors.txt or glove_vectors.txt. To see an example on how to load or use the embeddings, look at the bin/embedding_visualization.py script.

Reproducing paper's experiments

You can run the file run.sh. You'll need a few files, that you can generate as explained above. You can set your experiment options using a .env file in the root of the repo. Example:

ELABORATE_PER_MONTH=1
REMOVE_PRODUCTS_WITHOUT_DESC=0

GRAPH_FILE="./data/graph_2019-10-01_2019-11-01.pkl"
TRAJECTORIES_FILE="./data/trajectories_2019-10-01_2019-11-01.parquet"
GLOVE_VECTORS_FILE=./data/glove_emb_out_2019-10-01_2019-11-01/vectors.txt
GLOVE_VOCAB_FILE=./data/glove_emb_out_2019-10-01_2019-11-01/vocab.txt
PERCENTAGE_TRAJECTORIES_SAMPLE=0.01

EVAL_FREQUENCY=2048
TRAIN_TIMESTEPS=160000
NUM_EXPERIMENTS=32
NUM_TRAJECTORIES=16
SKIP_CONSECUTIVE_DUPLICATES=true
LEVENSHTEIN_REWARD_STRATEGY=plain
SAMPLING_STRATEGY=kmeans
USE_DAMERAU_LEVENSTH=true