Kiran Bhandari's blog

Kiran Bhandari's blog

#delta-loading

Articles tagged with #delta-loading

Delta Loading Using AWS Glue Job Bookmark and Dynamodb
There are few approaches to achieve the delta loading in AWS. I will be discussing more about the two option here; Using Glue Job bookmark Using dynamodb to read unprocessed file Glue Job Bookmark Using job bookmark, AWS glue tracks data that has...
Aug 17, 20243 min read143
Incremental Loading using Spark Partition
We can achieve the incremental loading using the spark data partition. I will be demonstrating with a simple example here, and you can you according to the business need. Let's consider the following data frame.(This could be your csv, or any formatt...
Jun 11, 20243 min read111

#delta-loading - Kiran Bhandari's blog