Data & AI · 2026
Real-Time Tweet Sentiment Analysis
An end-to-end big data pipeline that trains a sentiment classifier on 1.6M tweets with PySpark MLlib and serves live predictions with Kafka and Spark Structured Streaming.
What I did
- Trained a Logistic Regression model with PySpark MLlib on the Sentiment140 dataset (1.6M labeled tweets).
- A Kafka producer streams tweets into a Kafka topic; Spark Structured Streaming classifies them in real time.
- Predictions are stored in MongoDB and shown on a live dashboard through a Flask API.
- The whole system runs with Docker Compose.
Tech stack
- Python
- PySpark
- Spark Streaming
- Apache Kafka
- MongoDB
- Flask
- Docker Compose
Related projects
Next.jsHugging FaceTailwind CSS
Data & AI · 2025
AI Text Classifier
A Next.js app that classifies text against user-provided labels with Hugging Face's facebook/bart-large-mnli model, supporting Turkish and English.
Details →PythonHadoopHadoop Streaming
Data & AI · 2026
Iterative K-Means Clustering on MapReduce
K-Means clustering on a distributed Hadoop cluster with hand-written Python Mappers and Reducers via Hadoop Streaming, without Mahout or Spark.
Details →Apache PigPig LatinDocker Compose
Data & AI · 2026
Dockerized Apache Pig WordCount
Word counting and grouping with Apache Pig 0.17 in Docker, on data generated with Python.
Details →