Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port to Kotlin Multiplatform #25

Merged
merged 105 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
916052c
Ported Feature and BaseFeature to Kotlin
Ercalvez Nov 20, 2023
b98a883
Ported Feature and BaseFeature to Kotlin
Ercalvez Nov 23, 2023
f64ae87
Rename .java to .kt
Ercalvez Dec 9, 2023
b3cf2b2
kmp WIP, use okio for I/O
Ercalvez Dec 9, 2023
47c30a4
Rename .java to .kt
Ercalvez Dec 16, 2023
3b53d73
trying to fix unit test
Ercalvez Dec 16, 2023
37258f4
Fixed file access tests + Locale equality
Ercalvez Dec 17, 2023
788dfdb
Rename .java to .kt
Ercalvez Dec 17, 2023
28b420d
Fixed dissectCountryCode
Ercalvez Dec 17, 2023
b4aca73
Fixed dissectCountryCode
Ercalvez Dec 17, 2023
6b067f9
Merge remote-tracking branch 'origin/kmp' into kmp
Ercalvez Dec 17, 2023
145f846
Fixed remove useless trycatch
Ercalvez Dec 18, 2023
102c10a
Rename .java to .kt
Ercalvez Dec 20, 2023
d3a45d5
Converted all files to Kotlin
Ercalvez Dec 20, 2023
3832f45
Converted all files to Kotlin
Ercalvez Dec 20, 2023
efa97a9
Merge remote-tracking branch 'origin/kmp' into kmp
Ercalvez Jan 16, 2024
49f0c9b
Fix build
Ercalvez Jan 16, 2024
08b67d8
fix resource path
Ercalvez Feb 15, 2024
0355d0f
cleaned imports and unused files
Ercalvez Feb 18, 2024
86a375d
cleaned imports and unused files
Ercalvez Feb 18, 2024
fcdb098
Merge remote-tracking branch 'origin/kmp' into kmp
Ercalvez Feb 18, 2024
55c4d61
fixed getAll methods in tests
Ercalvez Feb 20, 2024
2382034
fix excludeCountryCodes unit tests + dependencies
Ercalvez Feb 22, 2024
e2d344b
fix excludeCountryCodes unit tests + dependencies
Ercalvez Feb 22, 2024
f71a920
fix merge conflict
Ercalvez Feb 24, 2024
7d7b1fd
fix build gradle
Ercalvez Feb 24, 2024
5a35637
Apply suggestions from code review
westnordost Feb 25, 2024
2dad8c1
Shorter code
westnordost Feb 25, 2024
3391252
add comment
westnordost Feb 25, 2024
8056008
make canonicalize a kotlin extension function
westnordost Feb 26, 2024
78b992b
add missing import
westnordost Feb 26, 2024
10f3e7c
fix doc comment
westnordost Feb 26, 2024
bba61db
simplify/kotlinize constructor
westnordost Feb 26, 2024
796301d
make startsWith parameter not nullable
westnordost Feb 26, 2024
b1103e9
improve readability:
westnordost Feb 26, 2024
a97473b
fix build (sorry, my bad)
westnordost Feb 26, 2024
0ead17f
make canonicalTerms non-nullable
westnordost Feb 26, 2024
4f54b2d
simplify constructor
westnordost Feb 26, 2024
704479b
remove equals and hashCode - data classes generate this automatically
westnordost Feb 26, 2024
1bb77dd
remove Builder - unnecessary, just using constructor will do
westnordost Feb 26, 2024
7a91894
simplify languageTag function
westnordost Feb 26, 2024
aa29df6
no need for Locale constants in production code
westnordost Feb 26, 2024
ecdb82c
format
westnordost Feb 26, 2024
19f5388
fix documentation comment
westnordost Feb 26, 2024
2502926
simplify constructor
westnordost Feb 26, 2024
a9ac25e
make non-nullable
westnordost Feb 26, 2024
ca1b7cd
make into extension functions for better readability
westnordost Feb 26, 2024
3428fff
use more descriptive variable names
westnordost Feb 26, 2024
5c2ce26
rename some variables, remove unnecessary type declarations
westnordost Feb 26, 2024
753ba34
remove TestUtils object, assertEqualsIgnoreOrder will report more hel…
westnordost Feb 26, 2024
6e6fb35
remove some unnecessary type declarations
westnordost Feb 26, 2024
7a956b9
BaseFeature is immutable
westnordost Feb 26, 2024
11d5088
remove unnecessary private vals from BaseFeature
westnordost Feb 26, 2024
ce1c5a7
make synchronizedGetOrCreate an extension function
westnordost Feb 27, 2024
e896893
correct indentation
westnordost Feb 27, 2024
fe5e4de
fixed indentation, removed unnecessary type declarations, use getOrPut
westnordost Feb 27, 2024
688d125
replace selector interface with function
westnordost Feb 27, 2024
f93f3a6
improve readability
westnordost Feb 27, 2024
9495b2f
improve readability, remove some unnecessary copying, operator get fu…
westnordost Feb 27, 2024
a72e383
fix LocalizedFeature
westnordost Feb 27, 2024
db13f9c
no need for assertEqualsIgnoreOrder helper function
westnordost Feb 27, 2024
c5934c2
fix operator access
westnordost Feb 27, 2024
74a313f
clean FeatureDictionary
westnordost Feb 27, 2024
6060392
move JVM specific tests to jvmTest
westnordost Feb 27, 2024
95bad31
remove Locale data class, use string instead
westnordost Feb 28, 2024
4de0ab7
make LocalizedFeature a data class
westnordost Feb 28, 2024
7b409ab
make classes that should be internal internal
westnordost Feb 28, 2024
fb55933
Merge remote-tracking branch 'origin/kmp-language-tag' into pr/25
westnordost Mar 1, 2024
b68f5c4
use ktor
Ercalvez Mar 4, 2024
0b1d7d7
Merge remote-tracking branch 'origin/kmp' into kmp
Ercalvez Mar 4, 2024
4ae4615
use ktor
Ercalvez Mar 4, 2024
293ba52
solve failing test
westnordost Mar 4, 2024
23ba651
synchronized map using `lazy` operator
Ercalvez Mar 8, 2024
52fad29
de.westnordost.osmfeatures -> de/westnordost/osmfeatures
Ercalvez Mar 23, 2024
fc67633
move android-specific stuff
westnordost May 10, 2024
b806f01
gradle shenanigans
westnordost May 10, 2024
6234dd3
move up
westnordost May 10, 2024
56fbda0
remove actually not supported platforms
westnordost May 10, 2024
6fe7b1a
formatting
westnordost May 10, 2024
a2dec4f
replace okio with kotlinx-io
westnordost May 10, 2024
5b9c5fc
correct synchronization / lazynization
westnordost May 10, 2024
0fec5c7
Classes from kotlinx-io-core are on the interface
westnordost May 10, 2024
46bc930
add entry for changelog
westnordost May 10, 2024
b7ecd80
fix tests
westnordost May 10, 2024
8316895
should be internal
westnordost May 10, 2024
6d1c037
fix android build
westnordost May 10, 2024
4b0b7a8
update readme
westnordost May 10, 2024
2004dc1
prettify parsers
westnordost May 10, 2024
0c99a57
more parser stuff
westnordost May 10, 2024
aeaee48
remove todo
westnordost May 10, 2024
71b7143
add test also for brand data
westnordost May 13, 2024
ab9037e
remove unnecessary map
westnordost May 13, 2024
29bbe6c
don't create a new regex every time that function is called
westnordost May 13, 2024
dad3413
naming
westnordost May 13, 2024
89f893a
rename file
westnordost May 13, 2024
7354d42
add test and fix
westnordost May 13, 2024
4fdd7a2
declare io exception can be thrown
westnordost May 13, 2024
5b82c63
make getters public
westnordost May 13, 2024
2f5c4e4
update readme
westnordost May 13, 2024
cc75a24
fix building names and terms index
westnordost May 13, 2024
3e4715a
fix tests
westnordost May 13, 2024
b6f3e0c
add link to ticket to monitor
westnordost May 14, 2024
e58ce94
is is more correct to use "language" rather than "locale"
westnordost May 14, 2024
6572300
builders should only be constructable from the methods of FeatureDict…
westnordost May 14, 2024
7e66378
Merge branch 'master' into kmp
westnordost May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

# 6.0

The library is now a Kotlin Multiplatform library. This is a breaking API change. There is also no separate artifact for Android anymore.

# 5.2

- Add support placeholders for preset names (breaking change in [v5.0.0](https://github.com/ideditor/schema-builder/blob/main/CHANGELOG.md#510) of iD presets schema)
Expand Down
136 changes: 66 additions & 70 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,105 +1,101 @@
# osmfeatures

A dictionary of OSM map features, accessible by terms and by tags, for Java and Android.
A Kotlin multiplatform dictionary of OSM map features, accessible by terms and by tags. Supported platforms are Android, JVM and iOS.

Requires Java 8.
Due to heavy use of indices, it is very fast.

## Copyright and License

© 2019-2022 Tobias Zwick. This library is released under the terms of the [Apache License Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.txt).
It is currently used in [StreetComplete](https://github.com/streetcomplete/streetcomplete).

## Installation

Add [`de.westnordost:osmfeatures:5.2`](https://mvnrepository.com/artifact/de.westnordost/osmfeatures/5.2) as a Maven dependency or download the jar from there.
## Copyright and License

For Android, use [`de.westnordost:osmfeatures-android:5.2`](https://mvnrepository.com/artifact/de.westnordost/osmfeatures-android/5.2).
© 2019-2024 Tobias Zwick. This library is released under the terms of the Apache License Version 2.0.

## Usage

Add [de.westnordost:osmfeatures:6.0](https://mvnrepository.com/artifact/de.westnordost/osmfeatures/6.0) as a Maven dependency or download the jar from there.

### Get the data

The data for the dictionary is not maintained in this repository.
It actually uses the [preset data from iD](https://github.com/openstreetmap/id-tagging-schema/blob/main/dist/presets.json) and [the translations of the presets](https://github.com/openstreetmap/id-tagging-schema/tree/main/dist/translations).
Just dump all the translations and the presets.json into the same directory.
Do not forget to give attribution to iD since you are using their data.

If you use gradle as your build tool, the easiest way to get this data is to put this task into your `build.gradle` and either execute this task manually from time to time or make the build process depend on it (by adding `preBuild.dependsOn(downloadPresets)`):

```groovy
import groovy.json.JsonSlurper

task downloadPresets {
doLast {
def targetDir = "$projectDir/path/to/where/the/data/should/go"
def presetsUrl = new URL("https://raw.githubusercontent.com/openstreetmap/id-tagging-schema/main/dist/presets.json")
def contentsUrl = new URL("https://api.github.com/repos/openstreetmap/id-tagging-schema/contents/dist/translations")

new File("$targetDir/presets.json").withOutputStream { it << presetsUrl.openStream() }

def slurper = new JsonSlurper()
slurper.parse(contentsUrl, "UTF-8").each {
if(it.type == "file") {
def filename = it.name.substring(0, it.name.indexOf("."))
def javaLanguageTag = Locale.forLanguageTag(filename.replace('@','-')).toLanguageTag()
def translationsUrl = new URL(it.download_url)
new File("$targetDir/${javaLanguageTag}.json").withOutputStream { it << translationsUrl.openStream() }
}
}
}
}
```
It actually uses the [preset data from iD](https://github.com/openstreetmap/id-tagging-schema/blob/main/dist/presets.json), [its translations](https://github.com/openstreetmap/id-tagging-schema/tree/main/dist/translations)
and optionally additionally the brand preset data from the [name suggestion index](https://github.com/osmlab/name-suggestion-index).
Each are &copy; iD contributors, licensed under the ISC license.


In StreetComplete (app that uses this library), there is [UpdatePresetsTask.kt](https://github.com/streetcomplete/StreetComplete/blob/master/buildSrc/src/main/java/UpdatePresetsTask.kt) to do this. It's longer but it is maintained and takes care of some more edge cases. Also, take note of [UpdateNsiPresetsTask.kt](https://github.com/streetcomplete/StreetComplete/blob/master/buildSrc/src/main/java/UpdateNsiPresetsTask.kt) which fetches the [name suggestion index presets](https://github.com/osmlab/name-suggestion-index) (=brands).
So, just dump all the translations and the presets.json into the same directory. To be always
up-to-date, it is advisable to have an automatic build task that fetches the current version of the
presets from the repository.

The app for which this library was developed (StreetComplete), uses the following tasks:
- [UpdatePresetsTask.kt](https://github.com/streetcomplete/StreetComplete/blob/master/buildSrc/src/main/java/UpdatePresetsTask.kt) to download presets and selected translations
- [UpdateNsiPresetsTask.kt](https://github.com/streetcomplete/StreetComplete/blob/master/buildSrc/src/main/java/UpdateNsiPresetsTask.kt) to download the brand presets from the [name suggestion index](https://github.com/osmlab/name-suggestion-index)

### Initialize dictionary

Point the dictionary to the directory where the data is located (see above). Use `FeatureDictionary` as a singleton.
```java
FeatureDictionary dictionary = FeatureDictionary.create("path/to/data"));
Point the dictionary to the directory where the data is located (see above). Use `FeatureDictionary` as a singleton, as initialization takes a moment (loading files, building indices).
```kotlin
val dictionary = FeatureDictionary.create(fileSystem, "path/to/data")
```

For Android, use
```java
FeatureDictionary dictionary = AndroidFeatureDictionary.create(assetManager, "path/within/assets/folder/to/data"));
```kotlin
val dictionary = FeatureDictionary.create(assetManager, "path/within/assets/folder/to/data")
```

If brand features from the [name suggestion index](https://github.com/osmlab/name-suggestion-index) (see last paragraph in the previous "Get the data" section) should be included in the dictionary, you can specify the path to these presets as a further parameter. These will be loaded on-demand depending on for which countries you search for.
If brand features from the [name suggestion index](https://github.com/osmlab/name-suggestion-index) should be included in the dictionary, you can specify the path to these presets as a third parameter. These will be loaded on-demand depending on for which countries you search for.

Translations will also be loaded on demand when first querying features using a certain language.

### Find matches by tags
```java
List<Feature> matches = dictionary
.byTags(Map.of("amenity", "bench")) // look for features that have the given tags
.forGeometry(GeometryType.POINT) // limit the search to features that may be points
.forLocale(Locale.GERMAN) // show results in German only, don't fall back to English or unlocalized results
.find();

// prints "Parkbank" (or something like this) or index out of bounds exception
// if no preset for amenity=bench exists that is localized to German
println(matches.get(0).getName());
```kotlin
val matches = dictionary.getByTags(
tags = mapOf("amenity" to "bench"), // look for features that have the given tags
languages = listOf("de"), // show results in German only, don't fall back to English or unlocalized results
geometry = GeometryType.POINT, // limit the search to features that may be points
)


// prints "Parkbank" (or something like this)
// or null if no preset for amenity=bench exists that is localized to German
println(matches[0]?.getName())
```

### Find matches by search word

```java
List<Feature> matches = dictionary
.byTerm("Bank") // look for features matching "Bank"
.forGeometry(GeometryType.AREA) // limit the search to features that may be areas
.forLocale(Locale.GERMAN, null) // show results in German or fall back to unlocalized results
```kotlin
val matches = dictionary.getByTerm(
term = "Bank", // look for features matching "Bank"
languages = listOf("de", null), // show results in German or fall back to unlocalized results
// (brand features are usually not localized)
.inCountry("DE") // also include things (brands) that only exist in Germany
.limit(10) // return at most 10 entries
.find();
// result list will have matches with at least amenity=bank, but not amenity=bench because it is a point-feature
country = "DE", // also include things (brands) that only exist in Germany
geometry = GeometryType.AREA, // limit the search to features that may be areas
)
// result sequence will have matches with at least amenity=bank, but not amenity=bench because it is a point-feature
// if the dictionary contains also brand presets, e.g. "Deutsche Bank" will certainly also be amongst the results
```

### Find by id

```kotlin
val match = dictionary.getById(
id = "amenity/bank",
languages = listOf("de", "en-US", null), // show results in German, otherwise fall back to American
// English or otherwise unlocalized results
country = "DE", // also include things (brands) that only exist in Germany
)
```

### Builders

For a more convenient interface on Java, the above functions continue to be available as builders,
e.g.

```java
Feature feature = dictionary
.byId("amenity/bank")
.forLocale(Locale.GERMAN, // show results in German, otherwise fall back to English etc.
Locale.ENGLISH,
null)
.inCountry("DE") // also include things (brands) that only exist in Germany
.get();
List<Feature> matches = dictionary
.byTags(Map.of("amenity", "bench"))
.forGeometry(GeometryType.POINT)
.inLanguage("de")
.find();
```

46 changes: 0 additions & 46 deletions build.gradle

This file was deleted.

126 changes: 126 additions & 0 deletions build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
plugins {
kotlin("multiplatform") version "1.9.24"
id("com.android.library") version "8.2.0"
id("org.jetbrains.dokka") version "1.9.20"

id("maven-publish")
id("signing")
}

kotlin {
group = "de.westnordost"
version = "6.0"

jvm()
androidTarget {
publishLibraryVariants("release")
compilations.all {
kotlinOptions {
jvmTarget = "1.8"
}
}
}
iosSimulatorArm64()
iosX64()
iosArm64()

applyDefaultHierarchyTemplate()

sourceSets {
commonMain {
dependencies {
// multiplatform file access
api("org.jetbrains.kotlinx:kotlinx-io-core:0.3.4")
// parsing the preset.json
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.6.3")
// for stripping diacritics correctly
implementation("com.doist.x:normalize:1.0.5")
}
}
commonTest {
dependencies {
implementation(kotlin("test"))
// we are pulling some current preset json from the iD preset repo to see if parsing
// does at least not crash and return something
implementation("io.ktor:ktor-client-core:2.3.11")
implementation("io.ktor:ktor-client-cio:2.3.11")
// ktor-client is a suspending API, so we need coroutines too
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.1")
}
}
}
}


android {
namespace = "de.westnordost.osmfeatures"
compileSdk = 33
defaultConfig {
minSdk = 21
}
}

val javadocJar = tasks.register<Jar>("javadocJar") {
archiveClassifier.set("javadoc")
from(tasks.dokkaHtml)
}

publishing {
publications {
withType<MavenPublication> {
artifactId = rootProject.name + if (name != "kotlinMultiplatform") "-$name" else ""
artifact(javadocJar)

pom {
name.set("osmfeatures")
description.set("Java library to translate OSM tags to and from localized names.")
url.set("https://github.com/westnordost/osmfeatures")

licenses {
license {
name.set("The Apache License, Version 2.0")
url.set("https://raw.githubusercontent.com/westnordost/osmfeatures/master/LICENSE")
}
}
issueManagement {
system.set("GitHub")
url.set("https://github.com/westnordost/osmfeatures/issues")
}
scm {
connection.set("https://github.com/westnordost/osmfeatures.git")
url.set("https://github.com/westnordost/osmfeatures")
}
developers {
developer {
id.set("westnordost")
name.set("Tobias Zwick")
email.set("[email protected]")
}
}
}
}
}
repositories {
maven {
name = "oss"
url = uri("https://oss.sonatype.org/service/local/staging/deploy/maven2/")
credentials {
val ossrhUsername: String by project
val ossrhPassword: String by project
username = ossrhUsername
password = ossrhPassword
}
}
}
}

signing {
sign(publishing.publications)
}


// FIXME - workaround for https://github.com/gradle/gradle/issues/26091
val signingTasks = tasks.withType<Sign>()
tasks.withType<AbstractPublishToMaven>().configureEach {
mustRunAfter(signingTasks)
}
23 changes: 4 additions & 19 deletions gradle.properties
Original file line number Diff line number Diff line change
@@ -1,22 +1,7 @@
# Project-wide Gradle settings.

# IDE (e.g. Android Studio) users:
# Gradle settings configured through the IDE *will override*
# any settings specified in this file.

# For more details on how to configure your build environment visit
# http://www.gradle.org/docs/current/userguide/build_environment.html

# Specifies the JVM arguments used for the daemon process.
# The setting is particularly useful for tweaking memory settings.
org.gradle.jvmargs=-Xmx1536m -Dfile.encoding=UTF-8

# When configured, Gradle will run in incubating parallel mode.
# This option should only be used with decoupled projects. More details, visit
# http://www.gradle.org/docs/current/userguide/multi_project_builds.html#sec:decoupled_projects
# org.gradle.parallel=true

systemProp.file.encoding=utf-8
# Gradle
org.gradle.jvmargs=-Xmx2048M -Dfile.encoding=UTF-8 -Dkotlin.daemon.jvm.options\="-Xmx2048M"
org.gradle.caching=true
org.gradle.configuration-cache=true

signing.keyId=08A11DBC
signing.password=
Expand Down
12 changes: 6 additions & 6 deletions gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#Thu Mar 04 22:04:30 CET 2021
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.3.3-all.zip
#Fri Dec 22 21:31:41 CET 2023
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.6-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
1 change: 0 additions & 1 deletion library-android/.gitignore

This file was deleted.

Loading