Scalable ETL Pipeline: Processing 5M+ retail records with PySpark on GCP Dataproc. Automated the extraction of global business KPIs and consumer trends. Includes an Ethical Data Framework to ensure privacy and fairness at scale

ecommerce GCP Streaming pipeline ― Cloud Storage, Compute Engine, Pub/Sub, Dataflow, Apache Beam, BigQuery and Tableau; GCP Batch pipeline ― Cloud Storage, Dataproc, PySpark, Cloud Spanner and Tableau

Monte Carlo stock simulation using Apache Spark.

Leveraging NYC Open Data, this repository contains Databricks notebooks for analyzing motor vehicle collisions. We perform EDA, spatial clustering, and predictive modeling on collision, vehicle, and person datasets to understand accident trends and predict potential risks.

Apache spark sandbox on GCP and Amazon EMR.

Project for Cloud Computing course (A.Y. 2018/2019)

Real-time stream processing project using Apache Kafka and Spark Streaming on Google Cloud Dataproc. Includes Python producers/consumers, Spark DStream word count, and full deployment with screenshots.

Implements a work queue for Dataproc Worflow Template executions

Movie Rating Analysis using Apache Spark (pyspark)

Hadoop Google DataProc DIO study

First project for Big Data course held at Roma Tre University

Comparative analysis of Apache Hive vs. Apache Spark on GCP Dataproc for analyzing 3.5M+ Disneyland attraction wait times and predictive modeling.

Car Insurance Cold Calls Data Analysis using Apache Hive

Marketing Campaign Data Analysis Using Apache Spark (PySpark)

Data is fetched from StackExchange, transformed using Pig, queried and stored in Hive. Additionally, the TF-IDF of the top 10 users is calculated using Hive.

Project for Scalable and Cloud Programming Course - 2018/19 UNIBO.

Process large amount of data and implement complex data analyses using Spark. The dataset has been made available by Google. It includes data about a cluster of 12500 machines, and the activity on this cluster during 29 days.