Sydney Airbnb Guest Satisfaction Classifier

Supervised ML pipeline that finds what actually drives guest satisfaction while debunking a misleading signal.

Role: Solo project / independent build
Stack: Python, scikit-learn, XGBoost, SHAP
Scope: 17,730 Sydney listings · Sep 2025 scrape

SHAP beeswarm summary plot showing host_scale_20+, amenities_count, and number_of_reviews as the strongest drivers of predicted guest satisfaction

The question I set out to answer sounds simple: what makes an Airbnb listing top-tier? But I wanted to answer it honestly, without the usual shortcuts. I built an end-to-end supervised ML pipeline on Inside Airbnb's Sydney dataset (17,730 listings, Sep 2025 scrape), classifying listings as top-tier vs. standard. Rather than picking an arbitrary percentile, I defined the target (review_scores_rating ≥ 4.9 with ≥ 5 reviews) through class-balance testing so the label meant something.

Early on I hit a wall: the price column was 100% null, and I verified that at both the listing and calendar level. Instead of dropping the dataset, I pivoted the feature strategy to host- and property-behaviour signals and documented the sourcing decision, because a real project rarely gives you the feature you expected.

Feature engineering was where the honesty mattered. I derived host tenure from signup dates, parsed amenities into counts and binary flags, and recovered bathrooms via regex, while deliberately excluding leakage columns like review sub-scores and host_is_superhost that would have let the model cheat. I compared Logistic Regression, Random Forest, and XGBoost, then tuned the winner with RandomizedSearchCV (3-fold CV) to reach ROC-AUC 0.771 on held-out test data.

Then SHAP (TreeExplainer) delivered the headline. The strongest driver of guest satisfaction wasn't price or amenities. It was host portfolio size: single-listing hosts hit a 46.6% top-tier rate versus just 16.2% for hosts managing 20+ listings. That one insight also explained a misleading instant-bookable pattern in the raw data. Instant-booking looked like it hurt satisfaction, but it was really just a marker for large operators who can't personally vet every stay. Two different methods, EDA and SHAP, landing on the same answer.

Project summary infographic: 17,730 raw listings cleaned and feature-engineered, model comparison by ROC-AUC, and the host-portfolio-size finding — The project at a glance: from 17,730 raw listings to a tuned, SHAP-explained model.

Confusion matrix on the held-out test set showing top-tier vs. not-top-tier predictions — Confusion matrix on the held-out test set: top-tier vs. standard listings.

Pythonscikit-learnXGBoostSHAPView on GitHub