Airbnb Availability Prediction Pipeline
Quera MLOps Minicamp
Built a pipeline for 10,480 Airbnb listings, tracked experiments in MLflow, and served single and batch predictions through a Dockerized FastAPI API.
Overview
In this project, I worked through a practical question. How can I combine data from several database tables, train a model, and serve predictions for new inputs through an API? My goal was to practice the full path from raw data to a reproducible prediction service, with a record of how I chose the model and a consistent set of inputs at training and serving time:
- I built and validated a dataset for 10,480 listings, using a cutoff to separate the inputs from the future calendar data that defined the label.
- I compared models in MLflow, deliberately tested a feature that leaked future information, and selected the random forest from the eligible runs.
- I served the selected pipeline through FastAPI, checked single and batch predictions, and compared two Docker builds.
The final service estimates whether an Airbnb listing will have low calendar availability over the next 30 days, using availability as a proxy for demand. A caller submits listing details and features derived from past reviews and calendar history. The API accepts one listing or a batch and returns a probability and a predicted class for each listing.
My Contribution
I built the PostgreSQL feature extraction and dataset validation, compared classifiers in MLflow, and implemented the model-loading and prediction logic for the FastAPI service. I packaged the service with Docker, ran it with Compose, and compared a full-base build with a smaller multi-stage build. Quera provided the shared PostgreSQL and MLflow services, assignment notebooks, and starter API schemas and tests.
How It Works
- Build the dataset. I fixed a cutoff date and assembled one row per listing from listing attributes, reviews, and 30- and 90-day calendar history. I derived the label from the following 30 days, audited personal fields, and validated the feature set before exporting CSV and Parquet.
- Compare models. I logged baselines, logistic regression variants, and a random forest in MLflow. One diagnostic run deliberately received a future availability feature that helped define the label. I excluded it from model selection and chose the random forest from the runs using only eligible features.
- Serve predictions. The FastAPI app loads the selected pipeline from MLflow at startup. It validates input, arranges features in training order, and returns probabilities and classes for single or batch requests. The packaging assignment also includes Kubernetes Deployment and Service manifests.
Data foundations
Two related exercises covered the steps before model development. I packaged a CSV pipeline for neighborhood summaries with Docker and DVC and pseudonymized fields in the input. In PostgreSQL, I compared the direct neighborhood summary query with a materialized view. The fastest read in five trials fell from 374 ms to 108 ms when reading the precomputed view.
Results and Scope
The validated dataset contained 10,480 listings with no duplicate listing rows at the cutoff and complete coverage for the 30-day label window. In a saved local run, the API returned two predictions in a single batch request.
The multi-stage build reduced local Docker image disk usage by about 53%, from 2.72 GB to 1.29 GB. The runtime image kept the application and installed dependencies on a slim Python base.
Task Repositories
- Build and validate the feature dataset for 10,480 listings. hw11_etl-pipeline
- Compare models in MLflow and select the random forest. hw12_mlflow
- Serve the selected model through a FastAPI API with single and batch predictions. hw13_fastapi-serving
- Package the service with Docker, compare image sizes, and define Kubernetes manifests. hw21_model-serving
- Package the CSV neighborhood-summary pipeline with Docker and DVC. hw01_airbnd-ops
- Benchmark the neighborhood summary query against a PostgreSQL materialized view. hw02_sql-metabase