you may use DBRX (databrick's LLM) or any other llm of your choice. just make sure you are using databricks in some part of your project (fine tuning, rag model, vector search, etc)
Do you guys provide Databricks resources to process large scale datasets e.g. GDELT Data is huge and it requires lot of computational resources to crawl, and needs lot of LLM calls to get insights required for use case. Please suggest/help us with Databricks resources to proceed further as this is a blocking issue for us at this moment. (Note: We also tried free tier Databricks but it has very limited offerings to help us with our blocking issues)
2 comments
Jackie Zhang Manager • about 2 years ago
you may use DBRX (databrick's LLM) or any other llm of your choice. just make sure you are using databricks in some part of your project (fine tuning, rag model, vector search, etc)
Akriti Dixit • about 2 years ago
Do you guys provide Databricks resources to process large scale datasets e.g. GDELT Data is huge and it requires lot of computational resources to crawl, and needs lot of LLM calls to get insights required for use case. Please suggest/help us with Databricks resources to proceed further as this is a blocking issue for us at this moment. (Note: We also tried free tier Databricks but it has very limited offerings to help us with our blocking issues)