Tag: Python

Duration

7 hours (usually 1 day including breaks)

Requirements

• A basic knowledge of Python
• A basic understanding of software engineering concepts
• Primary to intermediate level programming skills

Audience

• Programmers or developers with Python knowledge who want to develop multi-touch Android apps using the Kivy framework
• Android developers with Python knowledge

Overview

Kivy is an open-source cross-platform graphical user interface library written in Python, which allows multi-touch application development for a wide selection of devices.

In this instructor-led, live training participants will learn how to install and deploy Kivy on different platforms, customize and manipulate widgets, schedule, trigger and respond to events, modify graphics with multi-touching, resize the screen, package apps for Android, and more.

By the end of this training, participants will be able to

• Relate the Python code and the Kivy language.
• Have a solid understanding of how Kivy works and makes use of its most important elements such as, widgets, events, properties, graphics, etc.
• Seamlessly develop and deploy Android apps based on different business and design requirements.

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

Course Outline

To request a customized course outline for this training, please contact us.

Duration

28 hours (usually 4 days including breaks)

Requirements

• Basic computer skills: surfing websites, running programs, saving and opening documents, etc.
• A prior knowldege of programming is high recommended
• Knowledge of javascript frameworks like NodeJS is high recommended
• Some experience with Linux/Unix environment is preffered
• Good to have an open source technology exposure

Course Outline

Introduction to Python

• Controlling the flow of your program
• Working with lists
• Working with the dictionary data type
• Manipulating strings
• Pattern matching with regular expressions
• Reading, writing and managing files
• Debugging your code
• Pulling information from the internet (web scraping)
• Working with Excel, Word, and PDF Documents
• Working with CSV and JSON
• Keeping time
• Scheduling tasks
• Launching programs
• Sending emails and other messages
• Manipulating images
• GUI Automation

Introduction to Blockchain

• History and Background
• Companies using Blockchain
• Introdution to Bitcoin, Ethereum and Hyperledger
• Centralized v/s Decentralized v/s Distributed Networks
• Real Time Challenges

Bitcoin and Why are they Important

• History and Background
• Characteristics
• Transactions
• Mining
  • Hashing and Dual Key Encryption
  • Proof of Work
  • Merkle Tree
  • CPU v/s GPU mining
• Consensus

Ethereum

• Brief Background
• Technologies Behind
• Smart Contracts
• DAO
• Ether – The upcoming cryptocurrency

Blockchain for Development

• Ethereum Virtual Machine
• Introdution to Solidty
• Usage of Solidity IDE
• Setting up your first private Ethereum Network
  • go-ethereum
  • Genesis File
  • Solc Compiler
  • Remix IDE
• Connecting Nodes to Network
• Greeter
• Web3 Framework

Duration

21 hours (usually 3 days including breaks)

Requirements

Understanding of Database and statistics

Overview

Audience

*KDB+/Q developer

Format of the course

50% lectures, 40% labs, 10% tests

Course Outline

Backgrounds:

KDB+ is widely used in Financial industry and others. It is in-memory, column based, efficient especially procient tp process Financial data set. Many investbanks, hedged funds and prop-trading hours emploited KDB+ to many data analytics and data services. KDB+ play a significant role in analysis in back testing and daily trading, find out root cause and improve trading quality and efficiency.?Python is also widely used in Finiancial industry and it can manipulte KDB+ easily, provide many libs to do analysis.

In this course, will introduce how Q/KDB+/Python are used in Financial industries(how to store data, how is the data API used, how is gateway exploited to support concurrent connnections, trouble shooting and??support on KDB+ and etc) and many senarios and relevant solutions.

What’s the advantage of KDB+ in financial analysis?

– Senarios

– Performance & Efficiency

– which kind of financial dataset

KDB+ fundamentals

– type definiation & cast

– functional select/update/delete

– functions/lamda, sync/async function invocation

– web socket support

– file compression

– sym enumeration and denumeration

– splay table and partition

How can we deploy KDB+

– tickplant

– RDB/HDB

– gateway/API

– Reporting

How can we access KDB+

– Q

– Python

– R

– Java

– C/C++

How can import data from other data source into KDB+?

– txt/csv

– html/web page

– SQL Server

Duration

21 hours (usually 3 days including breaks)

Requirements

• Experience with PostgreSQL
• Python programming experience

Audience

• Developers
• Administrators

Overview

PostgreSQL (also known as Postgres) is an open source relational database management system or RDBMS built for handling large data workloads. Python is a dynamic and flexible programming language that supports various databases, such as MySQL, Oracle, PostgreSQL, etc.

This instructor-led, live training (online or onsite) is aimed at developers and administrators who wish to use PostgreSQL with Python to build secure, reliable, and stable applications at scale.

By the end of this training, participants will be able to:

• Install, configure, and connect PostgreSQL with Python.
• Get a refresher on Python programming fundamentals.
• Understand the benefits of using PostgreSQL with Python.
• Learn how to perform basic SQL operations with Python.
• Learn how to use the advanced PostgreSQL features with Python.
• Implement data safety, security, monitoring, and optimization techniques.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Course Outline

Introduction

• Overview of PostgreSQL features and architecture
• Python programming fundamentals
• Advantages of Python PostgreSQL 

Getting Started

• Installing PostgreSQL
• Installing Python, Django or Flask
• Installing psycopg2
• Connecting to PostgreSQL with Python

Python and SQL Operations

• The basics of SQL language
• Database schema and tables
• Writing a Python query
• Creating tables
• Inserting data
• Manipulating data
• Using limit clause
• Joining tables
• Full-text search
• Working with SQLite
• Examples of Python applications

Advanced PostgreSQL with Python

• Table inheritance
• Nested transactions
• User-defined types
• VIEWS, HAVING, DISTINCT
• Working with dates and times
• Multi-version concurrency control (MVCC)
• Charts, graphs, and reports using matplotlib

Administration and Security

• PostgreSQL data safety
• Restricting access using whitelist
• Implementing replication
• Database migration
• Backup and restore
• Monitoring queries
• Performance tuning
• Preventing SQL injection attacks

Troubleshooting

Summary and Next Steps

Duration

14 hours (usually 2 days including breaks)

Requirements

• An understanding of MongoDB’s document model 
• Python programming experience

Audience:
Developers

Overview

This instructor-led, live training (online or onsite) is aimed at developers wishing to learn how to use MongoDB as the database for Python their applications.

By the end of this training, participants will be able to:

• Install and configure MongoDB
• Understand the difference between accessing a NoSQL document database and a traditional relational databases’ (e.g., MySQL)
• Query a MongoDB database from within Python
• Create and write data to a MongoDB databasse
• Understand MongoDB’s “data processing pipeline”
• Perform real-time analytics and statistical analysis 
• Generate reports for dashboarding
• Implement exception handling in Python application

Format of the Course:

• Interactive lecture and discussion
• Lots of exercises and practice
• Hands-on implementation in a live-lab environment

Course Customization Options:

• To request a customized training for this course, please contact us to arrange.

Course Outline

Introduction
    Differences between MongoDB and MySQL, PostgreSQL, etc.

Setting up MongoDB

Overview of MongoDB Features and Architecture

• Mapping between a relational database and MongoDB
• Indexing, replication, load balancing, ad hoc querying, etc.

Connecting to MongoDB

Starting a Python + MongoDB application

Understanding the MongoDB Data Processing Pipeline

Reading and Writing to the database

Creating a New Database

Securing your Python Application

Understanding Availability in MongoDB

Scaling Your Application

• On-premise or public cloud

Troubleshooting MongoDB

Summary and Conclusion

Duration

14 hours (usually 2 days including breaks)

Requirements

• Familiarity with data analysis concepts

Audience

• Business analysts

Overview

In this instructor-led, live training, participants will learn three different approaches for accessing, analyzing and visualizing data. We start with an introduction to RDMS databases; the focus will be on accessing and querying an Oracle database using the SQL language. Then we look at strategies for accessing an RDMS database programmatically using the Python language. Finally, we look at how to visualize and present data graphically using TIBCO Spotfire. 

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Outline

======== Day 01 ========

Introduction to RDMS (Relational Database Management System) 

• Oracle RDMS and the SQL language

How Data is Stored in an RDMS

• Tables, records and columns

Connecting to an Oracle database

• Command line vs GUI
• Logging in: server, database, credentials

Data Analysis with SQL

• Querying the database using SELECT
• Filtering data with WHERE
• Sorting data with ORDER BY
• Querying multiple tables with Joins
• Grouping Data with GROUP BY
• Using aggregate functions to perform calculations on resulting data
• Filtering groups with HAVING

Introduction to Python

• Crash course on Python syntax

Connecting to an Oracle database

• Logging in: server, database, credentials

Data Analysis with Python

• Querying the database 
• Filtering, ordering, and counting data
• Advanced queries

======== Day 02 ========

Introduction to Data Visualization with TIBCO Spotfire

Connecting to a Spotfire Server

• Web client vs installed client

Importing Data

Data Visualization

• Bar Chart
• Line Charts
• Pie Chart

Filtering Data

Presentation Strategies with TIBCO Spotfire

Summary and Conclusion

Duration

35 hours (usually 5 days including breaks)

Requirements

• An understanding of Data Structure.
• Experience with Programming.

Audience

• Programmers
• Data Scientist
• Engineers

Overview

The training course will help the participants prepare for Web Application Development using Python Programming with Data Analytics. Such data visualization is a great tool for Top Management in decision making.

Course Outline

Day 1

1. Data Science
2. Data Science Team Composition (Data Scientist, Data Engineer, Data Visualizer, Process Owner)
3. Business Intelligence
   1. Types of Business Intelligence
   2. Developing Business Intelligence Tools
   3. Business Intelligence and the Data Visualization
4. Data Visualization
   1. Importance of Data Visualization
   2. The Visual Data Presentation
   3. The Data Visualization Tools (infographics, dials and gauges, geographic maps, sparklines, heat maps, and detailed bar, pie and fever charts)
   4. Painting by Numbers and Playing with Colors in Making Visual Stories
5. Activity

Day 2

1. Data Visualization in Python Programming
   1. Data Science with Python
   2. Review on Python Fundamentals

1. Variables and Data Types (str, numeric, sequence, mapping, set types, Boolean, binary, casting)
2. Operators, Lists, Tuples. Sets, Dictionaries
3. Conditional Statements
4. Functions, Lambda, Arrays, Classes, Objects, Inheritance, Iterators
5. Scope, Modules, Dates, JSON, RegEx, PIP
6. Try / Except, Command Input, String Formatting
7. File Handling
8. Activity

Day 3

1. Python and MySQL

1. Creating Database and Table
2. Manipulating Database (Insert, Select, Update, Delete, Where Statement, Order by)
3. Drop Table
4. Limit
5. Joining Tables
6. Removing List Duplicates
7. Reverse a String

2. Data Visualization with Python and MySQL
   1. Using Matplotlib (Basic Plotting)
   2. Dictionaries and Pandas
   3. Logic, Control Flow and Filtering
   4. Manipulating Graphs Properties (Font, Size, Color Scheme)
3. Activity

Day 4

1. Plotting Data in Different Graph Format
   • Histogram
   • Line
   • Bar
   • Box Plot
   • Pie Chart
   • Donut
   • Scatter Plot
   • Radar
   • Area
   • 2D / 3D Density Plot
   • Dendogram
   • Map (Bubble, Heat)
   • Stacked Chart
   • Venn Diagram
   • Seaborn
2. Activity

Day 5

1. Data Visualization with Python and MySQL
   1. Group Work: Create a Top Management Data Visualization Presentation Using ITDI Local ULIMS Data
   2. Presentation of Output

Duration

7 hours (usually 1 day including breaks)

Requirements

• Experience with Python and Apache Kafka
• Familiarity with stream-processing platforms

Audience

• Data engineers
• Data scientists
• Programmers

Overview

Apache Spark Streaming is a scalable, open source stream processing system that allows users to process real-time data from supported sources. Spark Streaming enables fault-tolerant processing of data streams.

This instructor-led, live training (online or onsite) is aimed at data engineers, data scientists, and programmers who wish to use Spark Streaming features in processing and analyzing real-time data.

By the end of this training, participants will be able to use Spark Streaming to process live data streams for use in databases, filesystems, and live dashboards.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Course Outline

Introduction

Overview of Spark Streaming Features and Architecture

• Supported data sources
• Core APIs

Preparing the Environment

• Dependencies
• Spark and streaming context
• Connecting to Kafka

Processing Messages

• Parsing inbound messages as JSON
• ETL processes
• Starting the streaming context

Performing a Windowed Stream Processing

• Slide interval
• Checkpoint delivery configuration
• Launching the environment

Prototyping the Processing Code

• Connecting to a Kafka topic
• Retrieving JSON from data source using Paw
• Variations and additional processing

Streaming the Code

• Job control variables
• Defining values to match
• Functions and conditions

Acquiring Stream Output

• Counters
• Kafka output (matched and non-matched)

Troubleshooting

Summary and Conclusion

Duration

7 hours (usually 1 day including breaks)

Requirements

• Experience with Python programming language
• Familiarity with stream-processing platforms

Audience

• Data engineers
• Data scientists
• Programmers

Overview

Apache Kafka is an open-source stream-processing platform that provides a fast, reliable, and low-latency platform for handling real-time data analytics. Apache Kafka can be integrated with available programming languages such as Python.

This instructor-led, live training (online or onsite) is aimed at data engineers, data scientists, and programmers who wish to use Apache Kafka features in data streaming with Python.

By the end of this training, participants will be able to use Apache Kafka to monitor and manage conditions in continuous data streams using Python programming.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Course Outline

Introduction

Overview of Apache Kafka Features and Architecture for Python

• Core APIs (producer, consumer, streams, connector)
• Concepts and uses

Accessing Kafka in Python

• Available Python libraries for use
• Compression formats supported

Installing Apache Kafka

• Computer installation
• Virtual private server and virtual machine installation

Starting Kafka Broker Server

• Reading and editing using an IDE (Integrated Development Environment)
• Running Zookeeper
• Logs folder

Creating a Kafka Topic

• Connecting to a Kafka cluster
• Reading topic details

Sending Messages Using Producers

• Initiating a producer
• Examining incoming messages
• Running multiple producers

Consuming Messages

• Kafka Console Consumer
• Running multiple consumers

Troubleshooting

Summary and Conclusion

Duration

28 hours (usually 4 days including breaks)

Requirements

• Experience with Python programming
• Basic familiarity with Hadoop

Overview

Hadoop is a popular Big Data processing framework. Python is a high-level programming language famous for its clear syntax and code readibility.

In this instructor-led, live training, participants will learn how to work with Hadoop, MapReduce, Pig, and Spark using Python as they step through multiple examples and use cases.

By the end of this training, participants will be able to:

• Understand the basic concepts behind Hadoop, MapReduce, Pig, and Spark
• Use Python with Hadoop Distributed File System (HDFS), MapReduce, Pig, and Spark
• Use Snakebite to programmatically access HDFS within Python
• Use mrjob to write MapReduce jobs in Python
• Write Spark programs with Python
• Extend the functionality of pig using Python UDFs
• Manage MapReduce jobs and Pig scripts using Luigi

Audience

• Developers
• IT Professionals

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

Course Outline

Introduction

Understanding Hadoop’s Architecture and Key Concepts

Understanding the Hadoop Distributed File System (HDFS)

• Overview of HDFS and its Architectural Design
• Interacting with HDFS
• Performing Basic File Operations on HDFS
• Overview of HDFS Command Reference
• Overview of Snakebite
• Installing Snakebite
• Using the Snakebite Client Library
• Using the CLI Client

Learning the MapReduce Programming Model with Python

• Overview of the MapReduce Programming Model
• Understanding Data Flow in the MapReduce Framework
  • Map
  • Shuffle and Sort
  • Reduce
• Using the Hadoop Streaming Utility
  • Understanding How the Hadoop Streaming Utility Works
  • Demo: Implementing the WordCount Application on Python
• Using the mrjob Library
  • Overview of mrjob
  • Installing mrjob
  • Demo: Implementing the WordCount Algorithm Using mrjob
  • Understanding How a MapReduce Job Written with the mrjob Library Works
  • Executing a MapReduce Application with mrjob
  • Hands-on: Computing Top Salaries Using mrjob

Learning Pig with Python

• Overview of Pig
• Demo: Implementing the WordCount Algorithm in Pig
• Configuring and Running Pig Scripts and Pig Statements
  • Using the Pig Execution Modes
  • Using the Pig Interactive Mode
  • Using the Pic Batch Mode
• Understanding the Basic Concepts of the Pig Latin Language
  • Using Statements
  • Loading Data
  • Transforming Data
  • Storing Data
• Extending Pig’s Functionality with Python UDFs
  • Registering a Python UDF File
  • Demo: A Simple Python UDF
  • Demo: String Manipulation Using Python UDF
  • Hands-on: Calculating the 10 Most Recent Movies Using Python UDF

Using Spark and PySpark

• Overview of Spark
• Demo: Implementing the WordCount Algorithm in PySpark
• Overview of PySpark
  • Using an Interactive Shell
  • Implementing Self-Contained Applications
• Working with Resilient Distributed Datasets (RDDs)
  • Creating RDDs from a Python Collection
  • Creating RDDs from Files
  • Implementing RDD Transformations
  • Implementing RDD Actions
• Hands-on: Implementing a Text Search Program for Movie Titles with PySpark

Managing Workflow with Python

• Overview of Apache Oozie and Luigi
• Installing Luigi
• Understanding Luigi Workflow Concepts
  • Tasks
  • Targets
  • Parameters
• Demo: Examining a Workflow that Implements the WordCount Algorithm
• Working with Hadoop Workflows that Control MapReduce and Pig Jobs
  • Using Luigi’s Configuration Files
  • Working with MapReduce in Luigi
  • Working with Pig in Luigi

Summary and Conclusion