Tag: distributed systems

Fluentd for Log Data Unification Training Course

Posted on by admin

Introduction

  • The need for distributed systems logging
  • The inadequacy of conventional logging solutions

Setting up Fluentd

Overview of Fluentd Features and Architecture

Configuration File Syntax

Overview of Event Workflow

Working with Fluentd Plugins

Overview of Fluentd Use Cases

Searching Data

Analyzing Data

Collecting Data

Archiving Data

Deploying Fluentd to Production

Logging and Monitoring

Managing Performance

Managing Plugins

Configuring Fluentd for High Availability

Troubleshooting

Summary and Conclusion

Which data storage to choose – from flat files, through SQL, NoSQL to massive distributed systems Training Course

Posted on by admin

Duration

7 hours (usually 1 day including breaks)

Requirements

Though no technical background is required, understanding the examples requires some level of database theory (e.g. SQL, etc…)

Overview

This course helps customer to chose the write data storage depend on their needs. It covers almost all possible modern approaches.

Course Outline

  1. File Document Storage (Cloud Storage)
    1. Features (OCR, Scalaibility, Search, etc…)
    2. Open Source examples (e.g. Next Cloud)
    3. Some commercial examples
  2. Flat file storage
    1. XML databases
    2. CSV databases
  3. Relational databases
    1. Normalization
    2. Dependencies and Constrants
    3. Scalability – replications, clusters
    4. Open Source and commercial software (MySQL, PostrgreSQL, DM7, Oracle, etc.)
  4. NoSQL Storage
    1. Document Oriented Databases (MongoDB, CouchDB etc…)
    2. Column Orientation (Canadra, Scylla etc…)
    3. Search Orientation (Elasticsearch…
  5. NewSQL
    1. CAP Theorem
    2. Opensource software (SequoiaDB, etc…)
  6. Search Engines
    1. Features (text processing, relevancy, etc…)
    2. Open Source examples
    3. Scalability, High Availability, Load Balacing, etc….
  7. Traditional Datawherehouses
    1. Business Inteligence, OLTP and Datawherehouse
    2. Opensource and commercial solutions
  8. MapReduce and Distributed Parallel Processing
    1. Hadoop-like (Hive, HFS, Impala)
  9. Distributed filesystem
    1. Overview of opensource (Ceph etc…)
  10. In-memory Databases
    1. Opensource solution (e.g. ApacheIgnite)
  11. Others
    1. Hypertable (Google Bigtable)
    2. BigQuery
    3. AWS solutsion (S3, etc…)
  12. Beyond present – future trends