Storing IOT Data in Azure: SQL vs Cosmos vs Other Methods

ChandanJha, what is the uses pattern for the data? Will we be performing real time analytics on the data? or we are planning to perform batch analytics on it? From the description, since we have FTP in the context, I am guessing we are performing batch analytics only. Also, from the discussion it is not very clear, if we really need to put the data into relational data store. Right?

If we need batch analytics and if there is no schema on write needed, then I would say we can bypass the IoT Hub and directly upload the data from FTP locations to Azure Data Lake Store Gen2, instead of Blob, unless you want to make IoT Hub as central messaging Hub. I see IoT Hub more in the real time ingestion and processing scenarios.

Please take a look at the architectural pattern here: 

• IoT reference architecture

The pattern will all depend on what you would like to do with the data downstream. In the above architecture, you can easily leverage ADLS Gen2, instead of Blob, in column 3, for cold path.

It sounds like you do not need hot path here. Granted IoT Hub gives you options to keep the messages until consumed in different ways (hot path/cold path etc.), I am not sure how complex it will be to work with IoT Hub to upload the data from FTP and whether it is considered a hot path.

Once your data is in ADLS Gen2, you can then leverage CosmosDB, to take advantage of unlimited scale, ms latencies, multi-regions writes etc.

Few other references:

IoT Device connectivity to Azure via FTP - Microsoft Tech Community

IoT Solutions and Azure Cosmos DB - Microsoft Tech Community

IoT using Cosmos DB

I hope this helps.

Thanks.

Meer Alam