Skip to content

Update introduction.md #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ShGanesh wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Update introduction.md #16

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/fairOS/introduction.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ The most important resources in a computer are the storage (disk) and compute (p

### Big Data Frameworks

During the turn of the last century, internet boomed and gave rise to many internet based applications. This helped companies to grow on a global scale, which in turn rise to data explosion and these companies started looking for Big Data Processing frameworks.
During the turn of the last century, the internet boomed and gave rise to many internet based applications. This helped companies to grow on a global scale, which in turn gave rise to data explosion, which led these companies to start looking for Big Data Processing frameworks.

It is not trivial to build distributed applications because of several factors. Hardware failures and algorithmic consensus are few of the problems that make it complex to build them. Because of that, several frameworks have emerged in the past which helped developers build applications that scale out. Most of them require the applications to be written in different paradigms and demand specific knowledge of the system. It is because of this, that the investment in writing a distributed application is high.

Expand All @@ -17,9 +17,9 @@ In the last couple of decades, many open source applications started to emerge c

![Big Data Framework Blueprint](/img/BigDataFrameWork.png)

The above diagram shows the basic blueprint of a Big Data Processing Framework. It contains two layers. The lower layer binds together the storage of all the computers involved, in to a Distributed File System (Ex: HDFS) and the upper layer taker care of the programming paradigm required to split the data and process it (ex: MapReduce). They usually have weak consensus which are non byzantine fault tolerant.
The above diagram shows the basic blueprint of a Big Data Processing Framework. It contains two layers. The lower layer binds together the storage of all the computers involved, in to a Distributed File System (Ex: HDFS) and the upper layer takes care of the programming paradigm required to split the data and process it (ex: MapReduce). They usually have weak consensus which are non byzantine fault tolerant.

The issues with this applications are, that following
The issues with these applications are,
- they give rise to data silos,
- they are not Byzantine fault tolerant,
- the data is owned by the organization and not by the user,
Expand All @@ -29,4 +29,4 @@ The issues with this applications are, that following

![Internet Computer](/img/InternetComputer.png)

The above diagram shows a blue print of an Internet Computer which has a Operating System that glues the web 3.0 storage layer with the already existing Big Data Frameworks. This requires that the storage layer is compatible with these frameworks and at the same time make the entire system Byzantine fault tolerant by introducing blockchain for consensus.
The above diagram shows a blue print of an Internet Computer which has a Operating System that glues the Web 3.0 storage layer with the already existing Big Data Frameworks. This requires the storage layer to be compatible with these frameworks and at the same time makes the entire system Byzantine fault tolerant by introducing blockchain for consensus.