dr_present 1

شكر و تقدير

TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE

FaaS(Food as a service )

ICT (Information and Communication Technology)

Cloud computing

Project objectives

•1 -

Project objectives2 .

Project objectives

3 .

Project objectives4 .

Project Achievements

Introduction to Cloud team

What, which, how and why?

• In the simplest terms, cloud computing means storing and accessing data and programs over the Internet instead of your computer's hard drive.

• When it's smarter to rent than to buy…

• Cloud computing is an emerging computing technology that uses the internet and central remote servers to maintain data and application.

But where does The Cloud come in??

Hardware

Operating System

App App App

Traditional StackHardware

OS

App App App

Hypervisor

OS OS

Virtualized Stack

Key Technology: Virtualization

A complex system consisting of complex subsystems

Architecture Cloud Computing

Hardware

CentOS

KVM

OpenStack

Big Data Web

Open Shift

Host OS. distro

Why centos?• Familiar of deploy and configure “recently released RDO”.

• Community support.

• Performance benchmarks “relating to updates”.

• Number of bugs.

Why GO open?• Multi-Vendor Virtual.

• Brings the efficiency, agility, and cost benefits of cloud.

• prevents one vendor from controlling your access to the greatest innovation

• Large supporting community.


Hardware

CentOS

KVM

OpenStack

Big Data Web

Open Shift

Virtualization


Hardware

CentOS

KVM

OpenStack

Big Data Web

Open Shift

OpenStack Architecture

Horizon

Nova

Neutron

SwiftCinder

Keystone

Glance

Ceilometer

Heat

OpenStack Installation• Preparation for installation:• Update OS.• Remove Network manager.• Add repositories.

Installation

Manual installation

Scripted installation

Installing services one by one “Manual installation”

• Service “keystone” 1. Installing the service: # yum install openstack-keystone python- keystoneclient

2. Configure the service:# openstack-config--set/etc/keystone/keystone.conf\ sqlconnectionmysql://keystone:KEYSTONE_DBPASS@controller/keystone

3. Start and enable Service#serviceopenstack-keystonestart #chkconfigopenstack-keystone on

• Service “glance”• Service “nova” • Enable networking• Add the dashboard “horizon”

OpenStack scripted deployment “RDO”

An automated installation with minimal user intervention. Answer file is a simple text file contains configurations of OS.Quick start:1. Getting software repositories.# yum update –y # yum install -y http://rdo.fedorapeople.org/rdo-release.rpm 2. Getting Packstack installer.# yum install -y openstack-packstack

3. Run Packstack to install openstack# packstack--allinone

OpenStack administration | cloud images

VM Image: Virtual disk that has a bootable operating system installed on it.

OpenStack supported image formats: RAW QCOW2: commonly used with the KVM hypervisor.

• The image size is smaller.• Support for snapshots.

VM Image metadata: CPU architecture. The OS installed on the image (Linux, Windows,… ). OS_Distro (Centos, Fedora, Ubuntu,…).

OpenStack Linux image requirements to be sure that the image supports the OpenStack features:

• Resize root partition on boot.• No hard-coded MAC address information.• Ensure ssh server runs.

• Access instance by using ssh public key.• Use cloud-init to fetch the public key. • Inject ssh key into the instance in the first boot.

• Process user data (host name, ssh key). Disable firewall and use the host firewall.

We built two cloud images: Centos cloud image: LAMP web server cloud Image:

• Linux operating system (Centos).• Apache HTTP Server is the most popular web server.• MySQL database.• PHP server-side scripting language designed for web.

Tools to modify images: Guestfish: modify the files inside image. Guestmount: mount the image's file system directly.

OpenStack administration | Internal Network Architecture

manage project. View and manage quotas. manage user. Manage flavors.

OpenStack administration | Admin user

OpenStack administration | End user

• Overview about menus.• Creating network.• Creating instances.• Setting IP’s.• Ping instances.


Hardware

CentOS

KVM

OpenStack

Big Data Web

Open Shift

What is Platform as a Service (PaaS)?

PaaS helps organizations, specifically developers, realize the benefits of cloud computing by providing a fast and scalable way to host applications in the cloud.

Technically, a PaaS is an Application Platform comprised of an operating system, middleware and other software that allows applications to run on the cloud with much of the management.

Platform as a Service is changing the way developers approach developing software.

What is OpenShift Origin? • OpenShift Origin is Red Hat’s open source Platform as a Service (PaaS)

offering.• OpenShift Origin is an application platform where application

developers and teams can build, test, deploy, and run their applications.

Code

Deploy

Enjoy

Flavors of OpenShift

origin

Public Cloud

Service

On-premise or Private

Cloud Software

Open Source Project

Modernizing App Development with PaaS

OpenShift Machines

Key OpenShift Terms• Broker – Management host, orchestration of Nodes• Node – Compute host containing Gears• Gear – Allocation of fixed memory, compute, and storage resources

for running applications• Cartridge – A technology/framework (PHP, Perl, Java/JEE, Ruby,

Python, MySQL, etc.) to build applications.

Simple Application Creation


Hardware

CentOS

KVM

OpenStack

Big Data Web

Open Shift

Big Data Architecture

Cloud Architecture

HDFS / MapReduce

Hive SqoopCloudera Platform

Eco Systems

Introduction to hadoop

What Makes it Big Data?

Introduction to hadoop

Big Data

How to store? How to work?

Hadoop environment

High level hadoop architecture

Introduction to HDFS

Introduction to MapReduce

The important thing in hadoop is configuration file

Practice work at lab


Cloud Architecture

HDFS / MapReduce

Hive Sqoop Cloudera Manager and New Solutions

Eco Systems

What’s the problem?

2 TB(compresed) raw data per day today

What’s the solution ?HIVE

Data Warehousing at Facebook Today

Web ServersScribe Servers

Filers

Hive on Hadoop Cluster

Oracle RAC Federated MySQL

HIVE Components

HIVE x BIG


Cloud Architecture

HDFS / MapReduce


Eco Systems

Sqoop

•Why Sqoop•What’s Sqoop

•How Sqoop works

Import data

1. Run Import2. Pull metadata

3. Lunch MapReduce job

4. Pull data from database

5. Writ to data sink

sqoop import --connect jdbc:mysql://vm2.faas.com/sqoop --username root --password root --table yelid --target-dir /input/

http://vm2.faas.com/sqoop


Export data

1. Run Sqoop Export

2. Launch MapReduce Job

3. Write to staging table

4. sql> INSERT INTO target (SELECT*FROM staging)

sqoop export --connect jdbc:mysql://vm2.faas.com/sqoop --username root --password root --table location --export-dir location




Cloud Architecture

HDFS / MapReduce


Eco Systems

What’s Cloudera CDH?• CDH : Is 100% Open Source Distribution including the core elements of Hadoop plus

several open source projects.

What's Inside ?• Apache Yarn : (Yet Another Resource Negotiator) Is the data operating system of Hadoop that enablesyou to process data simultaneously in multiple ways.

What's Inside?• Apache HUE : Hue is a suite of applications that provide web-based access to CDH

components and a platform for building custom applications.

What’s Cloudera Manager C.M?• Cloudera Manager : is a unified management interface that makes it easy to install, configure, and manage a CDH cluster through a web interface “Admin

Console”

Other Solutions1-Hortonworks : 100% Open Source Enterprise Apache Hadoop.

System Overview

Web Design and

Development

Big Data Solution

Cloud Computing

Web Design and Development

Our Next Step.!• Cloud: Integrate openShift with openStack.

• Big Data: Online data processing (real time) using HBase.

• Web: Automate matching between farmer and food processor.

Thank You

dr_present 1

Documents