Microsoft Sentinel’s Ingestion Shift: Moving from Legacy APIs to CCF and DCRs

← Home

Microsoft Sentinel is entering a cleaner, more Azure-native phase of log ingestion. The legacy HTTP Data Collector API is now out of support, and Microsoft is steering custom integrations toward the Logs Ingestion API and the Codeless Connector Framework (CCF).

For architects, the real story is not the retirement of one API. It is the shift away from custom ingestion plumbing toward DCR-driven, governed pipelines that are easier to secure, monitor and evolve. Many Sentinel estates still depend on scripts, Function Apps and direct HTTP calls built over several years. This change is a good opportunity to simplify that design rather than replace one endpoint with another.

What actually changes?

The legacy Data Collector API is now unsupported. Microsoft’s latest guidance is important here: this is not necessarily a hard technical shutdown on September 14. Existing TLS 1.2-compliant ingestion can continue, but the API is effectively in a retirement state and only receives critical security fixes. I would not treat that as a reason to delay migration.

The supported direction is DCR-based ingestion. The Logs Ingestion API gives us transformation capability, better control over the destination schema, support for multiple destinations and RBAC tied to the Data Collection Rule. For connector scenarios, CCF gives Microsoft Sentinel a more standardized way to build and operate integrations without carrying the same amount of custom code.

Legacy vs modern ingestion

The biggest difference is where responsibility sits. In the older pattern, we often own the polling logic, retries, authentication and transformation code ourselves. With CCF, much of the connector behaviour can be expressed declaratively, while DCRs become the control point for routing and transformation.

Where Blob Storage fits

One of the more interesting CCF patterns is Azure Storage Blob ingestion. A producer writes logs to Blob Storage, Event Grid signals new blobs through a Storage Queue, and the connector processes the data through a DCR into the destination table.

I like this pattern for sources that produce files or bursty volumes because the storage layer creates a useful buffer between the source and Sentinel. It is not required for every connector, but it gives architects another option when direct API polling is not the best fit.

How I would approach the migration

I would start by identifying every custom connector, Function App, Logic App or script that still posts directly to the Data Collector API. Then classify each one by source type and volume.

If the source is a straightforward custom application or service pushing logs, the Logs Ingestion API is usually the cleanest direction. If it is a product integration that needs polling, authentication and repeatable connector packaging, I would look closely at CCF. For file-based or high-volume batch scenarios, the Blob Storage CCF pattern may be worth considering.

The migration should also be used to revisit the DCR design: which fields are really needed, what can be filtered before ingestion, whether transformations belong in the DCR, and whether the destination table schema still makes sense. This is where the move can improve both security operations and ingestion cost.

My takeaway

The deadline matters, but the architectural opportunity matters more. Moving away from the legacy Data Collector API gives us a chance to reduce custom plumbing and put ingestion behind clearer Azure-native controls.

I would avoid a simple one-for-one API replacement. Treat this as an ingestion-design review: simplify the connector, move transformation and routing into DCRs where appropriate, use managed identity or Entra-based authentication, and make monitoring part of the design from day one.

Sources: Microsoft Sentinel data connectors · Migrate from the HTTP Data Collector API to the Logs Ingestion API · Microsoft Sentinel custom connector options

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top