Rui Pinto

I built an end-to-end batch pipeline using GCS, BigQuery, Terraform, Bruin, and Streamlit to turn raw GitHub events into analytics-ready insights. repo: github.com/RuiFSP/dezoomc… #DataEngineering #DEZoomcamp #DataTalksClub #Bruin #BigQuery #GCP

Module 7 complete: Stream Processing with Redpanda + PyFlink ✅ - Produced 49,416 Green Taxi Oct 2025 trips to Redpanda in ~3s - TUMBLE(5 min) window: top zone PULocationID 74 with 15 trip #DataEngineering #ApacheFlink #Redpanda #StreamProcessing

Module 6 complete: Batch Processing with Spark by @DataTalksClub Spark correctness depends on details like partitioning and timestamp handling, not only on writing transformation #DataEngineering #Spark #PySpark #BatchProcessing

Finished the @dltHub workshop from @DataTalksClub ✅ Built a custom API → dlt → DuckDB pipeline, with page-by-page pagination + validation. #dlthub #dezoomcamp #zoomcamp #dataengineering

📊 Module 5 of Data Engineering Zoomcamp done! - Data Platforms with Bruin - End-to-end ELT pipelines - Data quality & lineage - Deployment to BigQuery Free course by @DataTalksClub : github.com/DataTalksClub/…

Module 4 complete: Analytics Engineering with dbt ✅ Built data transformations: • 8 dbt models in 3 layers (staging → intermediate → core) • Hybrid approach: Local dbt Core CLI + Cloud BigQuery #DataEngineering #BigQuery #dbt @DataTalksClub

Module 3 complete: Data Warehousing & BigQuery ✅ Built optimized data warehouse: • 20.3M taxi records in BigQuery • Partitioning by date: 91% cost reduction (310MB → 27MB) • Clustering by VendorID for faster queries #DataEngineering #BigQuery @DataTalksClub

Module 2 of DE Zoomcamp by @DataTalksClub @wrussell1999 done! - @kestra_io workflow orchestration - ETL pipelines for taxi data - Backfill & scheduling - Variables & dynamic flows

Module 1 complete: Containerization & Infrastructure as Code ✅ @DataTalksClub • #Docker Compose orchestrating #PostgreSQL + pgAdmin • #Python ETL processing 46K NYC taxi records • #SQL analytics on trip patterns and zones • #Terraform provisioning GCP resources

Anthropic launched a free 9-chapter prompt-engineering masterclass with hands-on exercises, advanced techniques, and a GitHub playground perfect for improving AI prompting skills github.com/anthropics/pro…

After hundreds of Kaggle competitions and years of trial, error, and wins, our team @ChrisDeotte @The0Viel @0verfit @giba1, have compiled a playbook for tabular modeling—a system that’s carried us to the top of leaderboards and held up on real-world data. developer.nvidia.com/blog/the-kaggl…

Finished the Agents & MCP Workshop in #LLMZoomcamp @DataTalksClub - Learned about agentic RAG, function calling, and MCP for multi-agent orchestration. - Built agent tools & tested workflows #AI #Agents #RAG #MCP

Just finished the Data Ingestion Module Workshop from #LLMZoomcamp @DataTalksClub ✅ Built ELT pipelines with dlt ✅ Created a knowledge graph with Cognee ✅ Tagged concepts with Node Sets ✅ Made API docs searchable

MLOps Zoomcamp Module 6 on Best Practices! 🚀 @DataTalksClub Learned: ✅ Unit & integration testing with pytest ✅ Mocking cloud services with LocalStack ✅ Code quality with pre-commit hooks ✅ Workflow automation with Makefiles ✅ CI/CD with GitHub Actions #mlops #zoomcamp

📊 Week 2 of #LLMZoomcamp @DataTalksClub • Convert text to vector embeddings with FastEmbed • Calculate semantic similarity • Build a searchable vector db with @qdrant_engine • Compare different embedding models and dimensions • Optimize search by combining text fields

Just completed Module 1 of #LLMZoomcamp @DataTalksClub! 🚀 Built a complete RAG pipeline using Elasticsearch & OpenAI. Learned to: • Index & search documents with field boosting • Create context-aware prompts • Optimize for token usage & costs

✅ Module 5 of #MLOpsZoomcamp @DataTalksClub Built a full monitoring stack using Docker, Grafana, PostgreSQL, and Evidently. Tracked prediction drift on NYC taxi data, orchestrated with Prefect. Monitoring isn’t optional. It’s essential. #MLOps #MachineLearning #DataDrift

Just finished MLOps Zoomcamp Module 4: Deployment! 🚀 ✅ Built batch inference for NYC taxi trip predictions ✅ Containerized with Docker ✅ Added cloud storage integration ✅ Orchestrated with Prefect #MLOps #DataScience #MachineLearning

🚀 Just completed Module 3 of #MLOpsZoomcamp @DataTalksClub! ✅ Prefect for workflow orchestration ✅ Docker containers for reproducibility ✅ MLflow for experiment tracking One step closer to production-ready #MLOps skills! #DataScience #MachineLearning

✅ Finished MLOps Zoomcamp Module 2: Built NYC taxi trip prediction system with MLflow tracking, hyperopt tuning & model registry on SQLite backend. Thanks @DataTalksClub & @Al_Grigorev for this practical MLOps journey! #MLOps #DataScience #MLflow