AWS Big Data¶

Nubenetes V2 Elite Portal

You are browsing the AI-Curated V2 Elite Edition. Looking for the exhaustive list of references? Check out the V1 Historical Archive.

Architectural Context

Detailed reference for AWS Big Data in the context of Architectural Foundations.

Cloud Infrastructure¶

(2025) AWS Data Pipeline [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — Official page of AWS Glue, a fully managed serverless data integration service. Identifies structural discovery using AWS Glue Data Catalog, PySpark ETL execution, and schema registry controls.
(2025) AWS Data Pipeline Documentation [DOCUMENTATION] 🌟🌟 [COMMUNITY-TOOL] — Classic developer documentation for AWS Data Pipeline. Details the scheduling of batch workflows across EC2 and EMR resources (largely superseded by modern AWS Glue and Step Functions setups).
(2024) AWS Big Data Blog: Category - AWS Data Pipeline [DOCUMENTATION] 🌟🌟 [LEGACY] — Technical blog category detailing legacy deployment setups for AWS Data Pipeline, providing structural templates for data movements before cloud-native ETL tools gained dominance.

(2016) Querying Amazon Kinesis Streams Directly with SQL and Spark Streaming [SCALA CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Technical blueprint detailing Spark Streaming integration with Amazon Kinesis streams. Discusses record-processing window optimizations, checkpoint configurations, and SQL querying on live event pipelines.

(2021) whizlabs.com: AWS Kinesis vs Kafka Apache 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Comparative architectural review between AWS Kinesis and Apache Kafka. Analyzes data retention policies, throughput capabilities, scaling overheads, and total cost of ownership (TCO) profiles.

(2026) Amazon Aurora PostgreSQL now supports direct querying of Apache Iceberg and Parquet data in your data lake 🌟 [COMMUNITY-TOOL] — Technical documentation and implementation reference for Amazon Aurora PostgreSQL now supports direct querying of Apache Iceberg and Parquet data in your data lake covering AWS Data Lake in AWS (aws.amazon.com).
Building a Data Lake on AWS 🌟 [COMMUNITY-TOOL] — Technical documentation and implementation reference for Building a Data Lake on AWS covering AWS Data Lake in AWS (aws.amazon.com).

(2026) Amazon DynamoDB now supports real-time vector search at any scale 🌟 [COMMUNITY-TOOL] — Technical documentation and implementation reference for Amazon DynamoDB now supports real-time vector search at any scale covering Amazon DynamoDB in AWS (aws.amazon.com).

Using Spark SQL for ETL [COMMUNITY-TOOL] — Technical documentation and implementation reference for Using Spark SQL for ETL covering Introduction in AWS (aws.amazon.com).

💡 Explore Related: Chef | Xamarin | Monitoring