Python Tutorial #2

What exactly u and r – string flag means?

In Python 2, you can store the string in 2 different types.

The first one is ASCII which is str type in python, it uses 1 byte of memory. (256 characters, will store mostly English alphabets and simple symbols)

The 2nd type is UNICODE which is unicode type in python, it uses 2 bytes of memory. (65536 characters, so this include all characters of all languages on earth)

By default, python will prefer str type but if you want to store the string in unicode type you can put u in front of the text like u’text’ or you can do this by calling unicode(‘text’)

So u is just a short way to call a function to cast str to unicode. That’s it!

Now the r part, you put it in front of the text to tell the computer that the text is raw text, backslash should not be an escaping character. r’\n’ will not create a new line character. It’s just plain text containing 2 characters where first character is ‘\’ and second character is ‘n’

In [68]: a=’qmi\nasd’

In [69]: print(a)
qmi
asd

In [70]: a=r’qmi\nasd’

In [71]: print(a)
qmi\nasd

If you want to convert str to unicode and also put raw text in there, use ur because ru will raise an error.

NOW, the important part:

You cannot store one backslash by using r, it’s the only exception. So this code will produce an error: r’\’

To store a backslash (only one) you need to use ‘\’

If you want to store more than 1 characters you can still use r like r’\’ will produce 2 backslashes as you expected.

 

Advertisements

Python Tutorial #1

Here are some of the points which are important to me.

Regular expressions:

.*? means “match zero or more characters but as little as possible“. It is called a non-greedy match.

.* means almost the same except that as much as possible is matched (greedy match).

Examples:

  • Using f.*?a to match foo bar baz results in foo ba (stops after first a)
  • Using f.*a to match foo bar baz results in foo bar ba (stops after last a)

In Python regular expression, greedy means as much as possible.

Numeric Operator:

integer division,  a//b

float division,  a/b

Split Function: Separate two inputs with space by split function

i, c = input().split()

Difference between lists and tuples in python?

We can use tuples as

  1. keys in dictionaries
  2. elements of sets

whereas lists can not.

 

What is Data Science?

In today’s era, data is in many forms like audio, video, text files and in a large amount too. Due to this more than 80% of the data is unstructured. Data Science here comes in a role to make complex and analytical models to enhance useful information from this variety of data.

This useful information can not only predict the things but also can automate things like self-driving cars, pilotless aircraft. We can extract precise information for our customer’s requirements for business profit.

For more information, one can refer to the site

You can build your own Neural Network from scratch using this site

Below are the most common python libraries for data science with reference link.

Numpy

http://cs231n.github.io/python-numpy-tutorial/

https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html

Matplotlib

https://matplotlib.org/users/pyplot_tutorial.html

Install Anaconda

https://www.digitalocean.com/community/tutorials/how-to-install-anaconda-on-ubuntu-18-04-quickstart

Linear Regression Theory

https://www.internalpointers.com/post/linear-regression-one-variable

Multiple Linear Regression

https://datatofish.com/multiple-linear-regression-python/

Learning in Neural Networks

What is Learning?

Learning rule in Neural Network means machine learning i.e. how a machine learns. It is a mathematical logic to improve the performance of Artificial Neural Network.

Learning rule is one of the most important factors to decide the accuracy of ANN.

sc_learning

Synapse

Dictionary meaning of synapse is togetherness, conjunction. The purpose of the synapse is to pass the signals from one neuron to target neuron. It is the basis through which neurons interact with each other.

Presynaptic and Postsynaptic Neuron

Information flow is directional. The neuron which fires the chemical called neurotransmitter is presynaptic neuron and the neuron which receives neurotransmitter is postsynaptic neuron.

Learning Rules

sc_learning_1

Hebbian Learning

It is one of the oldest learning algorithms. A synapse(connection) between two neurons is strengthened if the neuron A on either side of the synapse is near enough to excite neuron B, and repeatedly or persistently takes part in firing it. It leads to some growth process or metabolic changes in one or both cells such that A’s efficiency as one of the cells firing B, is increased.

Generalization of Hebbian Rule

\,\Delta w_i = \eta x_i y,

The four properties which characterize Hebbian synapse are:-

Time-dependent mechanism:- In it, modification in the Hebbian Synapse depend on the exact time of the occurrence of the presynaptic and postsynaptic activities.

Local mechanism:- Since synapse holds information-bearing signals. This locally available information is used by Hebbian synapse to produce local synaptic modification that is input specific.

Interactive mechanism:- In it change in Hebbian synapse depends upon the activity levels on both sides of the synapse(i.e. presynaptic and postsynaptic activities).

Conjunctional or correlational mechanism:- The co-occurrence of presynaptic and postsynaptic activities is sufficient to produce synaptic modification. That is why it is referred to as a conjunctional synapse. The synaptic change also depends upon the co-relation between presynaptic and postsynaptic activities due to which it is called correlational synapse.

Competitive Learning

In competitive learning, the output neuron competes among themselves for being fired. Unlike Hebbian Learning, only one neuron can be active at a time. It is the form of unsupervised learning.

It plays an important role in the formation of topographic maps.

The three basic elements to a competitive learning rule:-

  1. A set of neurons that are same except some neurons and which therefore respond differently to the given input set.
  2. A limit imposed on the “strength” of each neuron.
  3. A mechanism that allows a neuron to compete among themselves for a given set of inputs. The winning neuron is called a winner-takes-all-neuron.

The internal activity vof the winning neuron must be the largest among all the neurons for a specified input pattern x. The output signal vof the winning neuron is set equal to one; the output signal of all other neurons that lose the competition is set to zero.

 

The change applied to synaptic weight wji is defined by

{\displaystyle \Delta w_{ij}=\left\{{\begin{matrix}\eta (x_{i}-w_{ij})&{\mbox{ if }}i=j\\0&{\mbox{ otherwise}}\end{matrix}}\right.}

sc_comp

 Boltzmann Learning

Boltzmann learning is statistical in nature. It is derived from the field of thermodynamics. It is similar to an error-correction learning rule. However, in Boltzmann learning, we take a difference between the probability distribution of the system instead of the direct difference between the actual value and desired output.

Boltzmann learning rule is slower than the error-correction learning rule because in it the state of each individual neuron, in addition to the system output is taken into account.

The neurons operate in a binary manner representing +1 for on state and -1 for off state. The machine is characterized by an energy function E

E=-\left(\sum _{{i<j}}w_{{ij}}\,s_{i}\,s_{j}+\sum _{i}\theta _{i}\,s_{i}\right)

where si is the state of neuron and wji is the synaptic weight between neuron i and j. The fact that means that none of the neurons in the machine has self-loopback. 

The probability of flipping the state of neuron j

p_\text{i=on} = \frac{1}{1+\exp(-\frac{\Delta E_i}{T})}

where  \Delta E_i is the energy change.

The neurons of Boltzmann machine is divided into two functional groups, visible and hidden. The visible neurons act as an interface between the network and the environment in which it operates, whereas the hidden neurons always act freely.

sc_boltzmann

The Boltzmann machine works on two modes of operation:

Clamped condition:- In this, the visible neurons are clamped onto the specific states. These states are determined by the environment.

Free-running condition:- In this, all the neurons including the visible neurons and hidden neurons are allowed to operate freely.

Boltzmann Learning uses only locally available observations under two modes of operations: clamped and free running.

Playing with Osmosis

Command to filter the file with tag amenity=school and write the filtered data in second file.

$ osmosis –read-xml file=”ludhiana.osm” –nkv keyValueList=”amenity.school” –write-xml file=”school.osm”

Do you want to update your OSM data without deleting existing data?

So here are the steps given below.

Step 1

Derive a change set between two files. The first file is the file after changing, the second planet file is the file before changing. The changeset will be written in the third file.

$ osmosis –read-xml file=”classroom.osm” –read-xml file=”class_lib_tcc.osm” –derive-change –write-xml-change file=”classroom_classlibtcc_diff.osm”

Step 2

Dump the changeset to the database in append mode. It means it will change only affected data. Rest will exist there.

$ osm2pgsql –slim -a -d classroom -C 2500 –hstore -S openstreetmap-carto.style ~/osmosis_files/class_audi_diff.osm

And you are done.:)

Note: Osmosis and osm2pgsql have different schemas. So for using postgis flags of osmosis we have to create schema compatible with osmosis. I followed this link to create ans setup database.

For my satisfaction, I have written that command again.

$ psql -d classroom -f /home/amisha/osm/osmosis/script/pgsnapshot_schema_0.6_action.sql

OUTPUT

CREATE TABLE
ALTER TABLE

 

Variety: Wallpaper Changer

Variety Software

Variety is an open source wallpaper changer for linux.

Features

  1. It allows to set your own wallpapers.
  2. Provide  an option to change wallpaper automatically on hourly, daily basis.

Installation

sudo add-apt-repository ppa:peterlevi/ppa
sudo apt-get update
sudo apt-get install variety

Open it either from dashboard or from terminal by running command “variety”. Set it according to your needs.

Screenlets: Desktop Widgets

The next thing to make your Ubuntu more beautiful is to display desktop widgets like clock, infoPanelDesktop in Ubuntu system with the help of Screenlets software using this link.

Wireshark: Packet Analyzer

Install Wireshark from this link.

Vim Configuration

Are you bored of using standard vim editor?

Do you really want to style and customise your vim editor? Then what you are waiting for.

I have configured vim using this link . The complete source code and documentation of vimrc file is written in my Github repo

Just follow the instruction given in the README.md file and impress your friends with it.;-)

Struggles:

After adding and installing plugin “YouCompleteMe” for auto-complete I was getting message while opening any file with vim.

“The ycmd server SHUT DOWN (restart with :YcmRestartServer)”

Even after running command :YcmRestartServer the problem stucked. After searcing on internet, I ran the python script “install.py” under “$HOME/.vim/bundle/YouCompleteMe”

The following table contains some of vim basic commands and the commands run for my vimrc file.

Command Description
Type v, yank lines then press Ctrl-c It will copy lines in visual mode. It will work if you copy this .vimrc file.
Tap Esc, then Ctrl -p It will paste lines in normal mode. It will work if you copy this .vimrc file.
:Te filename It will open that file within the vim editor. You can also select file instead of typing filename. For that type :Te and then open NerdTree(Ctrl +n) and select the exact file.
gt Switch between the files opened with :Te command.
Ctrl + ww Switch between the files opened with :Te command.
Ctrl-P It enables the search within the vim. It uses plugin ctrlp.vim.
:sp filename Split the layout vertically. It means the vim window is divided in two parts upper and lower.
:vs filename Split the layout horizontally. It means the vim window is divided in two parts left and right.
:b buffer_number Switch to that particular buffer.
:ls List all buffers. Buffers are the files open in that vim session.
0 To reach the beginning of a line.
$ To reach end of a line.
* Move to the next occurrence of the word under cursor.
# Move to the previous occurrence of the word under cursor.
gg To reach beginning of a file.
G To reach end of a file.
o To insert text into a new line.
dw Deletes the first word on the right side of the cursor.
. To repeat the previous command.
x To delete a character.
d$ Delete from cursor to the end of current line.
5yy Copy 5 lines.
line_no>>G To go to particular line number.
Yank lines,then type :e filename. Copy-paste from one file to other. Filename is the destination file where you want to paste the lines.

 

Bitcoins

What is Bitcoin?

Bitcoin is a new currency that was created in 2009 by an unknown person using the alias Satoshi Nakamoto and released as open-source software. It is a cryptocurrency and worldwide payment system. It is the first decentralized digital currency .

Transactions are made with no middle men – meaning, no banks! The price of bitcoin skyrocketed into the thousands in 2017.

Why Bitcoins?

With the help of bitcoins, international payments are easy and cheap because they are not tied to any country or subject to regulation.

Some people just buy bitcoins as an investment, hoping that they’ll go up in value.

Owning Bitcoins

 Bitcoins are stored in a “digital wallet,” which exists either in the cloud or on a user’s computer. Bitcoins are traded from one personal ‘wallet’ to another ( peer to peer communication).

Anonymity

Though each bitcoin transaction is recorded in a public log, names of buyers and sellers are never revealed – only their wallet IDs. That’s why it has become the currency of choice for people online buying drugs or other illicit activities.

Why bitcoins in demand?

1. It does not rely on a central bank.
2. Due to limited supply, people will pay more to get the coins they think have value.
3. It can be used for investment.

Future in question

No one knows what will become of bitcoin. It is mostly unregulated, but some countries like Japan, China and Australia have begun weighing regulations. Governments are concerned about taxation and their lack of control over the currency.