Databricks announced the acquisition of Fennel, a specialized platform focused on feature engineering for machine learning applications. This acquisition bolsters Databricks’ capabilities in feature engineering, especially for real-time and streaming data applications.

The acquisition also strengthens Databricks’ competitive position against rivals like Snowflake and cloud providers, while addressing a key pain point in the machine learning lifecycle.

Who is Fennel?

Fennel is a fully managed platform focused on simplifying feature engineering for machine learning applications. Founded in 2022, the company quickly established itself as a specialized solution in the feature engineering space. The founding team brings impressive credentials, having previously led AI infrastructure efforts at Meta and Google Brain.

Fennel’s core technology addresses a critical challenge in machine learning workflows: efficiently creating and managing features across both batch and real-time data streams. Its platform is designed to avoid redundant computations by only recomputing data that has changed, improving efficiency and reducing costs.

Notable customers include Upwork and Cricut, who use Fennel for various machine learning applications including credit risk decisioning, fraud detection, trust and safety systems, personalized ranking, and marketplace recommendations.

Strategic Fit within Databricks

The acquisition aligns with Databricks’ vision of creating a comprehensive Data Intelligence Platform. Fennel’s capabilities address several key challenges in the machine learning workflow:

1. Unified Batch and Real-Time Processing: Fennel’s ability to handle both batch and streaming data in a unified manner complements Databricks’ existing data processing capabilities.
2. Python-Native Experience: Fennel’s Python-centric approach aligns with Databricks’ commitment to providing accessible tools for data scientists, reducing reliance on specialized data engineering teams.
3. Incremental Computation: Fennel’s incremental computation engine, which only reprocesses data that has changed, brings efficiency improvements that will help Databricks customers reduce costs.
4. Improved Feature Management: The acquisition enhances Databricks’ ability to help customers quickly iterate on features and improve model performance with reliable signals.
5. Real-Time and Personalized Context for GenAI: Fennel’s capabilities will strengthen Databricks’ offering in providing fresh, personalized context for generative AI applications.

Analysis

The Databricks-Fennel acquisition ultimately highlights that despite the excitement around foundation models and generative AI, companies are still investing heavily in the fundamentals of machine learning infrastructure.

For Databricks customers, this acquisition promises more efficient, cost-effective feature engineering capabilities while strengthening Databricks’ position as a comprehensive data intelligence platform in a heavily competitive market.