osm4scala

High performance Scala library and Spark Polyglot (Scala, Python, SQL, etc.) connector for OpenStreetMap Pbf files.

Documentation and site

⚠ Full usage documentation at https://simplexspatial.github.io/osm4scala/

Stargazers over time

Dev information:

It's possible to develop using a Windows machine, but all documentation suppose that you are using Linux or Mac.

Prepare environment

The only special requirement is to execute sbt compile to generate the protobuf source code.

sbt compile

PATCH_211 flag

Because depending on the Scala version, there are projects that are disabled (No spark3 for Scala 2.11) and different libraries dependencies. Because this, there is a flag called PATCH_211 (default value is false) to enable or disable Scala 2.11 compatibility.

Cross versions

The project is using cross version to manage 2.11, 2.12 and 2.13 using the same code base, so remember to use '+' to trigger all versions versions.

So remember, as example, for testing:

PATCH_211=false sbt +test
PATCH_211=true sbt +test

Release process

The publication into Maven Central has been removed from the release process, so now there are few steps:

1. Release.

    git checkout master
    sbt release

2. Publish into Maven Central. Information about configuration in plugins involved:

   • sbt-pgp using gpg command-line utility under the cover.
   • Sonatype GPG documentation
   • xerial/sbt-sonatype
   • scala-lang documentation

   Basically:

   • Set the keys as sbt-pgp#working-with-pgp-signatures explains.
   • Be sure that public key has been uploaded into, at least last time, pgp.mit.edu: gpg --keyserver hkp://pgp.mit.edu --send-keys <key>
   • Set the right credentials file at $HOME/.sbt/1.0/sonatype.sbt.
   • Execute:

     git checkout v1.*.*
     sbt clean
     PATCH_211=false sbt +publishSigned
     PATCH_211=true sbt +publishSigned
     # In this point, tree target/sonatype-staging/ will show all artifacts to publish.
     sbt sonatypeBundleRelease

3. Publish documentation and site.

    git checkout v1.*.*
    cd website
    nvm use
    export GIT_USER=<username>; export USE_SSH=true; npm run deploy

References.

PBF information:

• PBF2 Documentation: http://wiki.openstreetmap.org/wiki/PBF_Format
• PBF2 Java library: https://github.com/openstreetmap/osmosis/tree/master/osmosis-osm-binary
• Download whole planet pbf files: http://free.nchc.org.tw/osm.planet/
• Download country pbf files: http://download.geofabrik.de/index.html
• Scala protocol buffer library: https://scalapb.github.io/ and https://github.com/thesamet/sbt-protoc
• OSM primitives: https://wiki.openstreetmap.org/wiki/Elements
  • Node: https://wiki.openstreetmap.org/wiki/Node
  • Way: https://wiki.openstreetmap.org/wiki/Way
  • Relations: https://wiki.openstreetmap.org/wiki/Relation
  • Tags: https://wiki.openstreetmap.org/wiki/Tags

third party OSS libraries:

• ScalaPB: https://scalapb.github.io/ and https://github.com/thesamet/sbt-protoc