Osmosis in OSM

Till now, whatever changes I do in the OSM data, I again download the latest data from OSM or from http://extract.bbbike.org/ and import the fresh data in the database. But this is not the efficient approach.

So Today, I am going to introduce you with the term osmosis. You can read it in detail from here

Osmosis Installation

$sudo apt install osmosis

$osmosis -v                          //tell version

INFO: Osmosis Version 0.44.1

Now, it’s time to use the tool. The first command I tried to check the difference in the changesets of the two files was

$ osmosis –read-xml file=”ludh.osm” –read-xml file=”ludhiana.pbf” –derive-change –write-xml-change file=”planetdiff-1-2.osc”

Luckily, I encountered with an error.;)

SEVERE: Thread for task 1-read-xml failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error has occurred, aborting.

After browsing on the internet then I came to know about my silly mistake. For the file ludhiana.pbf I was using the flag –read-xml instead of –read-pbf. The correct command is

$ osmosis –read-xml file=”ludh.osm” –read-pbf file=”ludhiana.pbf” –derive-change –write-xml-change file=”planetdiff-1-2.osc”

Similarly, we can explore osmosis a lot.

 

 

FEM Presentation 2

As I have already mentioned in my previous blog I am taking course on Finite Element Method. I have given presentation on the topic Galerkin Form or Finite Dimensional Weak Form. I don’t know how much fruitful it was to the audience over there.

But the one who is reading this blog and was present over there can comment and questions for the same.

The documentation for the same is stored in zoho docs

After that I have put the whole code of webmap in my Github repository under the branch webmap.

https://github.com/amisha2016/pbOSM/tree/webmap

Now the task I am going to do is how to update data of OSM rather import it from scratch again and again. What I think it will be done through osmosis. But I am not sure.

Today, I tried was trying to make wiki page for pbOSM project and from that I came to to know that it is made with the Markup language. Again I have to learn one new language.:(

Now, it’s time to actualize the thing. So, bye bye for a time being.

FEM Saturday Begins

It’s necessary to nomenclature this Saturday as the master goal of the presentation was same  for all trainees “Finite Element Method Discussion”.

Necessity of this discussion?

Inspite of having very good videos by Coursera our focus is to assemble individuals knowledge at a hot point(TCC) and make a cool script out of it.;)

As I am having Jio at home. So, it’s difficult for me to watch videos online. For that  I tried to download whole playlists from youtube through youtube-dl command.

To download any playlist there is a special URL for that which is having playlist keyword in it. For example https://www.youtube.com/playlist?list=PLJhG_d-Sp_JHKVRhfTgDqbic_4MHpltXZ

This URL can be found by clicking on the link as shown below highlighted by red marker.

imageedit_4_5865450091

Command to download whole playlist is

$youtube-dl –yes-playlist –continue https://www.youtube.com/playlist?list=PLJhG_d-Sp_JHKVRhfTgDqbic_4MHpltXZ

While downloading big playlists if it breaks in between due to any technical issue like Net connection, Power off etc. you may use –continue flag to download existing playlists.

Apart from this I got a sticker of Apertium

IMG_1127.JPG

Apertium is free open source software for machine translation. For example

Screenshot from 2017-07-30 15-58-30.png

 

Inreased Zoom Level to 28 in OSM

Do you want to increase the zoom level of the OSM server? Then my friends, you are at right blog.;)

For it I am assuming that you have already made your pc an OSM server. While doing so, you have encountered with the term mod_tile. Am I right?

So, the whole story will revolve around mod_tile. If you are excited to know more about mod_tile then you can browse to my previous blogs.

Coming back to the topic we have to change overall 4 files in the directory mod_tile.

Edit the file /go/to/your/mod_tile/directory/openstreetmap-tiles-update-expire

EXPIRY_MAXZOOM=18

to

EXPIRY_MAXZOOM=28

The second file you have to edit is renderd.conf.

In [default] section uncomment the line MAXZOOM=18 and the change the value  from 18 to 28.

Note:- The file renderd.conf is located in two places. First one in mod_tile directory and other one in /usr/local/etc. You have to make changes in mod_tile directory because after making all necessary changes and recompiling mod_tile again. This renderd.conf file in mod_tile directory will automatically cloned to /usr/loca/etc.

Now, the turn comes for the third file i.e. mod_tile/includes/render_config.h

Change the variable

#define MAX_ZOOM 18

to

#define MAX_ZOOM 28

The last file which you have to edit is mod_tile/src/store_file_utils.c

This file need a lot of changes. So better to backup of this file initially.

 unsigned char i, hash[5];
to
unsigned char i, hash[7];

for (i=0; i<5; i++) {
to
for (i=0; i<7; i++) {

snprintf(path, len, “%s/%s/%d/%u/%u/%u/%u/%u.png”, tile_dir, xmlconfig, z, hash[4], hash[3], hash[2], hash[1], hash[0]);
to
snprintf(path, len, “%s/%s/%d/%u/%u/%u/%u/%u/%u/%u.png”, tile_dir, xmlconfig, z, hash[6], hash[5], hash[4], hash[3], hash[2], hash[1], hash[0]);

int i, n, hash[5], x, y, z;
to
int i, n, hash[7], x, y, z;

n = sscanf(path+i, “/%40[^/]/%d/%d/%d/%d/%d/%d”, xmlconfig, pz, &hash[0], &hash[1], &hash[2], &hash[3], &hash[4]);
if (n != 7) {
to
n = sscanf(path+i, “/%40[^/]/%d/%d/%d/%d/%d/%d/%d/%d”, xmlconfig, pz, &hash[0], &hash[1], &hash[2], &hash[3], &hash[4], &hash[5], &hash[6]);
if (n != 9) {

for (i=0; i<5; i++) {
to
for (i=0; i<7; i++) {

if (n != 4) {
to
if (n != 6) {

unsigned char i, hash[5], offset, mask;
to
unsigned char i, hash[7], offset, mask;

for (i=0; i<5; i++) {
to
for (i=0; i<7; i++) {

snprintf(path, len, “%s/%s/%d/%u/%u/%u/%u/%u.%s.meta”, tile_dir, xmlconfig, z, hash[4], hash[3], hash[2], hash[1], hash[0], options);
to
snprintf(path, len, “%s/%s/%d/%u/%u/%u/%u/%u/%u/%u.%s.meta”, tile_dir, xmlconfig, z, hash[6], hash[5], hash[4], hash[3], hash[2], hash[1], hash[0], options);

snprintf(path, len, “%s/%s/%d/%u/%u/%u/%u/%u.meta”, tile_dir, xmlconfig, z, hash[4], hash[3], hash[2], hash[1], hash[0]);
to
snprintf(path, len, “%s/%s/%d/%u/%u/%u/%u/%u/%u/%u.meta”, tile_dir, xmlconfig, z, hash[6], hash[5], hash[4], hash[3], hash[2], hash[1], hash[0]);

You have noticed that the hash value hash[5] is converted to hash[7]. It is due to make the metatile path longer. Not understood yet? Let me give an example.

The metatile take 5 bytes= 40 bits to cover all tiles for maximum zoom level 20. In simple words /var/lib/mod_tile/a/b/c/d/e/.meta

20 bits for x axis and 20 bits for y axis.

For zoom level 20 it require 5 bytes. Correspondingly for zoom level 28 it require 7 bytes.

28 bits for x axis and 28 bits for y axis.

That is why hash[5] changed to hash[7].

Now recompile your mod_tile again through the below commands.

make

sudo make install

sudo make install-mod_tile

sudo ldconfig

Now change the maxzooom parameter to 28 in Leaflet or any other Javascript library. You are done with it.

If you are having any problem with it. Then please do comment.

Data Structure- Relations

The basic data structure to map OSM are

  1. Nodes
  2. Ways
  3. Relations

To understand relations you should know about the first two elements. A Node is the x-y coordinate(basically a point) which is used to represent a point, icon on the map.

The table in the database which represent this nodes is planet_osm_nodes. This table defines the latitude, longitude of the point

Talking about the another element is “Ways” which is nothing but set of coordinates or a nodes to make a line, arc. In OSM it is used to represent rivers, boundary of a region etc.

The table in the database which represent these ways are planet_osm_ways. It displays unique id of a way, which nodes are participating to make a way and tags associated with the way.

Relations are an important data structure of OSM. Relations are basically combination of any of the two or all of the elements(Nodes, Ways, Relation). Relations are the conceptual database Model.

There are various types of relation like multipolygon, routes etc. To know more about relation types browse to http://wiki.openstreetmap.org/wiki/Types_of_relation

Invalid Multipolygons

  1. Unclosed Polygon like endpoint of way 2 and 3 are not joined.

400px-Multipolygon_Illustration_INVALID_1.svg

  1. If the endpoint of a way share more than two ways. As shown in fig endpoints of way #2 is shared by way #3, #1, #4 .

400px-Multipolygon_Illustration_INVALID_2.svg

Now, here I explain the table planet_osm_rels. There are columns id, rel_off, parts, members, tags.

  1. id– It assigns unique id to the osm relation. It can’t be null.
  2. rels_off– It tell number of members(nodes, ways, relation in total) participating to make a relation.
  3. parts– It tell which elements are participating by telling their unique id.
  4. members– It tell about the name of the element and associated role with it. Now here role describe the function of a member. Like, if type of relation is multipolygon than role of member(way) is inner or outer.

I think with this we have got a lot of overview of the data structures used in OSM.

Playing with HTML

Today, I created the HTML page having a big table for Testing and Consultancy Cell billing system(for GST purpose). TCC use Librehatti software for placing orders and lot more.

https://amisha.gdy.club/libre.html

It is still getting modified.

The full source code is available here-> https://github.com/GreatDevelopers/LibreHatti

It is made by Great Developers Group only. For the first time I used this software and is quite appealing to me.

This HTML page is an excellent example of colspan and rowspan attributes used in table attribute.

Initially, I haven’t done indentation of my code. Then Ranvir told me to do so. For that he guide me to use Atom editor. No doubt, I have listened about it before also but didn’t get a chance to try it. It’s easy to download.

Procedure for Ubuntu 16.04 LTS

$ sudo add-apt-repository ppa:webupd8team/atom
$ sudo apt update; sudo apt install atom

 

Integration of 3D-OSMBuilding with Leaflet and OpenLayers

H ye Friends.

I think this is my first blog in which I will discuss about 3-D OSM Map with you all. Yes, it’s really nice project.

You can view it over here-> http://osmbuildings.org/

The source code of the project is https://github.com/OSMBuildings/OSMBuildings/

Now, here comes a turn what I have done in this project.

I have tried to add my own base map layer (i.e. 2-D Map )which is rendering the customize styles.

For it you have to edit index.html file in it. Change the old line to new line.

osmb.addMapTiles(‘https://{s}.tiles.mapbox.com/v3/osmbuildings.kbpalbpk/{z}/{x}/{y}.png’);                                                                                                      //Old Line

osmb.addMapTiles(‘../pb_osm/{z}/{x}/{y}.png’);                                //New Line

The other task which I have done which resembles with the blog title is Integeration of OSMBuildings with Leaflet.

You can clone it from here-> https://github.com/amisha2016/leaflet-3DMap

You can view it over https://lab.gdy.club/leaflet-3DMap/leaf.html. Isn’t it looking pretty;)

To do for OpenLayers you can refer to https://github.com/bogdanvaduva/osmbuildings

Mod_tile Installation Confirmation

Today, I got to know very interesting thing. To check whether mod_tile is working properly or not. You can browse to the “your_ip_address/mod_tile”. If it gives  you stats then Congratulations!

Screenshot from 2017-07-09 00-53-45

Apart from it, today was a seminar day. I got to know interesting things about OpenLayers Library i.e spin effect, pan effect, take a tour etc.

To explore it more you should browse to Abhinav blog

Track CPU and Memory Usage and Time Command

Today, I tried to explore and customize Ubuntu.

Track CPU and Memory Usage in Ubuntu Using Indicator-Sysmonitor

It’s clearly explained in the following link.

https://www.maketecheasier.com/track-cpu-memory-indicator-sysmonitor/

Done.

Then to note time of consumption by a particular command. You can use command

$time <your command>

Example:- $ time renderd -f -c /usr/local/etc/renderd.conf

Output

real 9m34.716s
user 0m16.796s
sys 0m0.960s

Done.

For the first time I used bitbucket. It’s versioning control system.

For it install mercurial.

$ sudo apt install mercurial
$ hg clone https://bitbucket.org/lawgon/osmindia

 

 

Admin Levels at Different Zoom Levels

Few days back, I got a task to show admin_levels at various zoom levels. I here explain the task in detail.

It means like to show all states (admin_level=4) of India at particular zoom level with a different colour. Likewise to show all districts (having a particular admin_level) with another colour. It was quite interesting task.

To know more about admin_levels you can browse to the link.

http://wiki.openstreetmap.org/wiki/Tag:boundary%3Dadministrative

The algorithm behind it is to give the reference to particular admin level. For example column “admin_level_2” will refer to all the rows where admin_level=2.

For this you must know basics of Postgresql.

– id: “zoom_levels”
name: “zoom_levels”
class: “”
geometry: “polygon”
<<: *extents
Datasource:
<<: *osm2pgsql
table: |-
(SELECT boundary,
COALESCE(tags->’name:pa’,tags->’name:hi’,name) as name, way, COALESCE(
‘admin_level_’ || CASE WHEN admin_level IN (‘2’, ‘3’,’4′,’5′,’6′,’7′,’8′,’9′,’10’) THEN admin_level ELSE NULL END
) as admin_level, access,
religion,
way_area/NULLIF(!pixel_width!::real*!pixel_height!::real,0) AS way_pixels
FROM planet_osm_polygon
WHERE admin_level IN (‘2’, ‘3’,’4′,’5′,’6′,’7′,’8′,’9′,’10’)
) AS zoom_levels
properties:
minzoom: 3

The above is the complete layer with id “zoom_levels”. The styles to this layer will be given through this ID.

Now create fresh mss file called zoom-levels.mss

#zoom_levels {
[admin_level = ‘admin_level_2’][zoom>=3][zoom<5] {
text-name: “[name]”;
text-line-spacing: -1.3;
text-size:12;
text-halo-radius: @standard-halo-radius;
text-fill: #000;
text-face-name: @book-fonts;
line-color: #000;
polygon-fill: #765543;
polygon-gamma:0.4;
polygon-opacity: 0.5;}
[admin_level = ‘admin_level_3’][zoom>=5][zoom<6] {
polygon-fill: @landform-color * 0.1;
polygon-gamma:0.4;
line-color: #000;
polygon-opacity: 0.65;
text-name: “[name]”;
text-size:12;
text-line-spacing: -1.3;
text-halo-radius: @standard-halo-radius;
text-fill: #000;
text-face-name: @book-fonts;}
[admin_level = ‘admin_level_4’][zoom>=6][zoom<8]{
polygon-fill: #cccccc;
polygon-gamma:0.4;
line-color: #000;
polygon-opacity: 0.65;
text-name: “[name]”;
text-size:16;
text-line-spacing: -1.65;
text-halo-radius: @standard-halo-radius;

text-fill: #000;
text-face-name: @book-fonts;}
[admin_level = ‘admin_level_5’][zoom>=8][zoom<10] {
polygon-fill: #ccc149;
polygon-gamma:0.4;
line-color: #000;
polygon-opacity: 0.65;
text-name: “[name]”;
text-size:11;
text-line-spacing: -1.3;
text-halo-radius: @standard-halo-radius;
text-fill: #000;
text-face-name: @book-fonts;}
[admin_level = ‘admin_level_6’][zoom>=10][zoom<12] {
polygon-fill: #cc1039;
polygon-gamma:0.4;
line-color: #000;
polygon-opacity: 0.65;
text-name: “[name]”;
text-size:14;
text-line-spacing: -1.3;
text-halo-radius: @standard-halo-radius;
text-fill: #000;
text-face-name: @book-fonts;}
[admin_level = ‘admin_level_7’][zoom>=12][zoom<14] {
polygon-fill: #1e7b79;
polygon-gamma:0.4;
line-color: #000;
polygon-opacity: 0.65;
text-name: “[name]”;
text-size:13;
text-line-spacing: -1.3;
text-halo-radius: @standard-halo-radius;
text-fill: #000;
text-face-name: @book-fonts;}
[admin_level = ‘admin_level_8’][zoom>=14][zoom<16] {
polygon-fill: #6c7b6d;
polygon-gamma:0.4;
line-color: #000;
polygon-opacity: 0.65;
text-name: “[name]”;
text-size:13;
text-line-spacing: -1.3;
text-halo-radius: @standard-halo-radius;
text-fill: #000;
text-face-name: @book-fonts;}
[admin_level = ‘admin_level_9’][zoom>=16][zoom<17] {
polygon-fill: #d68035;
polygon-gamma:0.4;
line-color: #000;
polygon-opacity: 0.5;
text-name: “[name]”;
text-size:13;
text-line-spacing: -1.3;
text-halo-radius: @standard-halo-radius;
text-fill: #000;
text-face-name: @book-fonts;}
[admin_level = ‘admin_level_10’][zoom>=17][zoom<19] {
polygon-fill: #295332;
polygon-gamma:0.4;
line-color: #000;
polygon-opacity: 0.5;
text-name: “[name]”;

text-size:12;
text-line-spacing: -1.3;
text-halo-radius: @standard-halo-radius;
text-fill: #000;
text-face-name: @book-fonts

}
}

Include this mss file in the project.mml file as other mss files are included.

Now it’s time to compile the code.:)

carto project.mml > style.xml

Re-render the tiles. You may see the desired output at admin_level=4. It’s obvious that you should have India database.

admin.png