Big Data

Big Data

Currently you can save, organize and analyze large amounts of data easily. And, briefly, this is the concept of Big Data. You have a huge amount of data and can organize this data and analyze them in order to obtain information that matters in useful time.
The Big Data [Big Data in 2013] main features are volume (amount of data), velocity (speed of data in and out) and variety (range of data types and sources), called the 3V's of Big Date. The volume considers large amounts of data growing exponentially. The velocity with which working data must be reduced. Information must be obtained in useful time. The characteristic of the variety is related to data formats. The data can be in various formats, from binary files to XML structures, JSON,etc ...
But there are two important aspects implicit in the concept Big Data that can not be bleached, the veracity and the value. Veracity is an important aspect because no matter have volume, speed and variety if the data is not reliable. The value is the aspect that justifies all the work and investment in Big Data, the value of information obtained from large data storage.

Relational databases have little flexibility are therefore not suitable for working with Big Data. It is here you enter the NoSQL (NotOnlySQL) that can work with large and growing volume of information.
The NoSQL solutions can be:

  • Storage key / value that can be represented as a hashmap or an associative array.
  • Super storage columns where the information is stored in the form of columns without relationships.
  • document storage functions similarly to the key / value but the value is a format, for example, BSON or JSON.
  • Storage graphs where data are represented by nodes or arcs and are related.
  • oriented storage objects where the data is stored as objects.
The way information is accessed and manipulated varies from solution to solution. Some NoSQL databases are MongoDB, Cassandra, Redis, Hbase, BigTable and others.

However some relational databases already incorporate NoSQL, such as MySQL and PostgreSQL, thereby exploring the best of both worlds. With relational databases bearing semi-extruturados data allows the user to use NoSQL document storage and maintain the ACID (atomicity, consistency, isolation and durability) set of properties that ensure security for database transactions.
Two very important concepts and that has to be addressed when discussing this subject is the MapReduce. and sharding.
The mapreduce, basically, is a design pattern that maps the input data and produces a list of key / value.
The process of storing data records across multiple machines is called sharding. Makes data storage management maintaining a uniform storage in systems with several machines to store data. Sharding suport data growth and the demands of read and write operations.


Regular Expressions

Regular Expressions

Let's talk about regular expressions and show some examples of regular expression implementations.
Regular expressions are a concise and flexible way to identify certain strings. To validate data entry or do search and information extraction in texts. Normal sequences used on license plates Portuguese system, according to wikipedia, are: AA-00-00 or 00-AA-00 or 00-00-AA
To identify these patterns using regular expressions we can use the following:

[A-Z] {2} - [0-9]} {2 - [0-9] {2}
[0-9] {2} - [A-Z] {2} - [0-9] {2}
[0-9] {2} - [0-9] {2} - [A-Z] {2}

Another practical example is to find three distinct sequences, as a real number, a real number with percentage a number in scientific notation.
Thus to identify a real number and counting as valid numbers with leading zeros and numbers only zeros to the right of the decimal point such as 002.0 or even 0.0 is used the following regular expression:
- [0-9] + [0-9] +
To identify a real number with percentage just add the sign '%':
- [0-9] + [0-9] +%
And, considering that for scientific notation can be represented with an 'e' case-sensitive and is always raised to a positive or negative integer is:
- [0-9] + [0-9] + [ee] - [0-9] +

Another example currently very useful, is to use regular expressions to validate email addresses. To validate emails will consider the following rules:

    You must start with an alphanumeric character
    Before @ alphanumeric and can have the following three symbols (-. _)
    After the @ equal but is mandatory to have a point
    After this point can have 2, 3 or 4 letters.
So is the following expression:
[A-Za-z0-9] + ([A-Za-z0-9 _.-]) * [@] [A-Za-z0-9] + ([A-Za-z0-9 _.-]) * [.] [A-Za-z] {2,4}


Free / Libre and Open Source Software

Once all the software which I speak here will be or Open-Source Software (OSS) or Free Software I believe it is important to dedicate a post to this topic. Not that I have anything against proprietary software, it is simply because I have several things against proprietary software.

The differences between Open-Source Software and Free Software are many and in a technical, philosophical and political context. Briefly open source software is a development methodology and free software is a social movement. The free software community, considers the free software as an ethical imperative, only free software respects the freedom of users. The philosophy of open source community considers issues in terms of how to make software "better" - and only in a practical sense. The main differences are in the software licenses. The free software allows you to change and adapt the code of spontaneous manner without the need to request permission from its owner.
However, these two software groups unite in various situations as Free / Libre and Open Source Software (FLOSS) and allow us to take advantage of software often with quality superior to proprietary software. And for free.

Already begin to see changes in large systems, migration to free software in order to save millions of euros in some cases and avoid situations of legal complications. For examples the insurance company case. Has migrated operating system to Linux. And the case of a teacher in Russia that was sued by Microsoft to use windows without a license.

To be at home for optimos basiados operating systems on Linux, for there is libreoffice office, to surf the internet firefox or chrome, the GIMP for image editing, Thunderbird to manage emails. VLC to view video. All legal, good and free.


Geographic Information Systems

Geographic Information Systems

Aronoff (1989) described a GIS as any manual or computer-based set of procedures to store and manipulate geographically referenced data.

GIS ‘Geographical Information System’ is an computer-based tool for mapping and analyzing things that exist and events that happen on Earth.
A GIS consists of:
Digital Data –– the geographical information that you will view and analyse using computer hardware and software.
Computer Hardware –– computers used for storing data, displaying graphics and processing data.
Computer Software –– computer programs that run on the computer hardware and allow you to work with digital data. A software program that forms part of the GIS is called a GIS Application.

With a GIS application you can open digital maps on your computer, create new spatial information to add to a map, create printed maps customized to your needs and perform spatial analysis.

GIS has significant importance in various professional areas such as urban planners, Geologists, foresters and others, to provide organized geographic information and allow analyze this information.




Opera Mobile Store Will replace the Nokia Store

The partnership between Microsoft, Nokia and Opera now appears to be taking a step further with the passage of the Nokia applications store for the control of Opera. This new agreement aims to ensure that all entry-level phones from Microsoft are served with an application store that can meet their needs and is prepared for the future offer the best deals. The need to exchange shop lies in the fact that Microsoft want to "disappear" Nokia all their products and do not want to keep the current store of Nokia. The new store will serve all equipments range of Microsoft, and all models of the Series 40, Series 60, Symbian. The migration of users from the current store for the Opera will begin soon and should be completed at the first half of 2015. Then, the app store will be closed from Nokia.


Subscribe to ZXCoders Blog RSS