Finding a new home can be a nightmare.
All of the logistics can be overwhelming: the showings to book, the nonstop emails to review, the loan application paperwork, inspections and more. But even before all of that, just sifting through the listings loses its luster around the one-thousandth click — especially if it’s a specific aesthetic that you are after.
Maybe the dream home non-negotiables are things like a bay window or a sink built into the kitchen island. While most home search websites have at least a few features built into the filters — like a fenced-in yard, basement, water view, etc. — there are rarely ones that can check off everything someone wants in a home.
The machine learning engineers at Realtor.com took this as a challenge — game on. Now entering the ring: a “homes with similar rooms” search feature.
Realtor.com decided to take advantage of OpenAI’s Contrastive Language-Image Pre-training open-source model. A tech blog on the company’s website explained that “while CLIP can be fine-tuned to identify specific image attributes with extreme accuracy, it also performs quite effectively out of the box at identifying attributes it has not been explicitly trained to recognize.”
Meaning: it does a good job of separating out visual data in an image — and when combined with segmenting each image as a separate vector based on the area of the house it represents, the result was a new feature that quickly saw success.
The engineering team saw more page views and found that users were particularly interested in finding homes with similar kitchens. Today, the team is able to use information like this to adjust and iterate the product to better serve the user’s needs.
Built In spoke with Ryan Rollings, a principal machine learning engineer at Realtor.com, about how machine learning has been at the heart of the product from the beginning, and the competitive advantage that gives the team in where they go next.
How is your team integrating AI and ML into the product development process, and what specific improvements have you seen as a result?
We’re using AI to augment our day-to-day workflows like everyone else, but one of my favorite applications is creating ad-hoc analyses of user feedback. In an earlier era of ML, we would spend weeks to months developing tools to summarize customer feedback, and even then, I could only produce a list of words that required thorough interpretation to make sense of. Now, product and engineering stakeholders are summarizing and exploring user feedback with custom prompts on the fly.
Can you share some examples of how AI/ML has directly contributed to enhancing your product line or accelerating time to market?
We help people find homes and connect with real estate agents who guide them through their home-buying, selling and renting journeys. At the heart of our platform is advanced search and recommendations technology, powered by AI/ML. We’ve evolved through multiple waves of ML sophistication — from basic statistical models, to deep learning, to foundation models.
For years now we’ve enhanced our search experience with intelligent listing insights from both image and text. Now, we’re raising the bar by developing LLM-driven solutions designed with robust safety and ethics guardrails ensuring that our next stage provides meaningful, trustworthy value for our users.
“We’ve evolved through multiple waves of ML sophistication … mow, we’re raising the bar by developing LLM-driven solutions designed with robust safety and ethics guardrails ensuring that our next stage provides meaningful, trustworthy value for our users.”
What strategies are you employing to ensure that your systems and processes keep up with the rapid advancements in AI and ML?
I love the hackathon culture at Realtor.com. There’s nothing like a demo to flesh out an idea and inspire teams to tackle a project. We use hackathons as a way to encourage continuous learning and curiosity, and many of our AI/ML innovations have come from these demos.
Throughout the year, we collect and share what we’re learning during well-attended weekly checkpoints. We also work with our cloud providers and generative AI platform hosts to gain early access for research on production-ready capabilities. Beyond what I learn from my colleagues, I regularly enhance my own awareness by reading newsletters and listening to podcasts I’ve collected over time.
