scala-uri

scala-uri is a small Scala library that helps you work with URIs. It has the following features:

A RFC 3986 compliant parser to parse URLs and URNs from Strings
URL Builders to create URLs from scratch
Ability to transform query strings with methods such as filterQuery and mapQuery
Ability to replace and remove query string parameters
Ability to extract TLDs and public suffixes such as .com and .co.uk from hosts
Ability to render URLs in punycode
Ability to parse IPv6 and IPv4 addresses
Support for custom encoding such as encoding spaces as pluses
Support for protocol relative urls
Support for user information e.g. ftp://user:[email protected]
Support for URNs
Support for mailto URLs
Support for data URLs as defined in RFC2397
Support for git scp-like URLs
Support for Scala.js
Support for cats
No dependencies on existing web frameworks

To include it in your SBT project from maven central:

"io.lemonlabs" %% "scala-uri" % "4.0.4"

There are also demo projects for both scala and Scala.js to help you get up and running quickly.

Parsing

Parse a URL

import io.lemonlabs.uri.Url

val url = Url.parse("https://www.scala-lang.org")

The returned value has type Url with an underlying implementation of AbsoluteUrl, RelativeUrl, UrlWithoutAuthority, ProtocolRelativeUrl or DataUrl. If you know your URL will always be one of these types, you can use the following parse methods to get a more specific return type

import io.lemonlabs.uri._

val absoluteUrl = AbsoluteUrl.parse("https://www.scala-lang.org")
val relativeUrl = RelativeUrl.parse("/index.html")
val mailtoUrl = UrlWithoutAuthority.parse("mailto:[email protected]")
val protocolRelativeUrl = ProtocolRelativeUrl.parse("//www.scala-lang.org")
val dataUrl = DataUrl.parse("data:text/plain;base64,SGVsbG8sIFdvcmxkIQ%3D%3D")

Note: scala-uri only supports parsing port numbers less than Int.MaxValue, deviating from RFC3986 which does not impose a limit

Parse a URN

import io.lemonlabs.uri.Urn

val urn = Urn.parse("urn:isbn:0981531687")
urn.scheme // This is "urn"
urn.nid // This is "isbn"
urn.nss // This is "0981531687"

Parse a URI

You can use Uri.parse to parse URNs as well as URLs. Url.parse and Urn.parse are preferable as they return a more specific return type

Building URLs

Url provides an apply method with a bunch of optional parameters that can be used to build URLs

import io.lemonlabs.uri.{Url, QueryString}

val url = Url(scheme = "http", host = "lemonlabs.io", path = "/opensource")
val url2 = Url(path = "/opensource", query = QueryString.fromPairs("param1" -> "a", "param2" -> "b"))

Transforming URLs

mapQuery

The mapQuery method will transform the Query String of a URI by applying the specified PartialFunction to each Query String Parameter. Any parameters not matched in the PartialFunction will be left as-is.

import io.lemonlabs.uri.Url

val uri = Url.parse("/scala-uri?p1=one&p2=2&p3=true")

// Results in /scala-uri?p1_map=one_map&p2_map=2_map&p3_map=true_map
uri.mapQuery {
  case (n, Some(v)) => (n + "_map", Some(v + "_map"))
}

The mapQueryNames and mapQueryValues provide a more convenient way to transform just Query Parameter names or values

import io.lemonlabs.uri.Url

val uri = Url.parse("/scala-uri?p1=one&p2=2&p3=true")

uri.mapQueryNames(_.toUpperCase) // Results in /scala-uri?P1_map=one&P2=2&P3=true
uri.mapQueryValues(_.replace("true", "false")) // Results in /scala-uri?p1=one&p2=2&p3=false

filterQuery

The filterQuery method will remove any Query String Parameters for which the provided Function returns false

import io.lemonlabs.uri.Url

val uri = Url.parse("/scala-uri?p1=one&p2=2&p3=true")

// Results in /scala-uri?p2=2
uri.filterQuery {
  case (n, v) => n.contains("2") && v.contains("2")
}

uri.filterQuery(_._1 == "p1") // Results in /scala-uri?p1=one

The filterQueryNames and filterQueryValues provide a more convenient way to filter just by Query Parameter name or value

import io.lemonlabs.uri.Url

val uri = Url.parse("/scala-uri?p1=one&p2=2&p3=true")

uri.filterQueryNames(_ > "p1") // Results in /scala-uri?p2=2&p3=true
uri.filterQueryValues(_.length == 1) // Results in /scala-uri?p2=2

collectQuery

The collectQuery method will transform the Query String of a URI by applying the specified PartialFunction to each Query String Parameter. Any parameters not matched in the PartialFunction will be removed.

import io.lemonlabs.uri.Url

val uri = Url.parse("/scala-uri?p1=one&p2=2&p3=true")

// Results in /scala-uri?p1_map=one_map
uri.collectQuery {
  case ("p1", Some(v)) => ("p1_map", Some(v + "_map"))
}

Convert an Absolute URL to a Relative URL

import io.lemonlabs.uri.Url

val absoluteUrl = Url.parse("http://www.example.com/example?a=b")
absoluteUrl.toRelativeUrl // This is /example?a=b

Convert a Relative URL to an Absolute URL

import io.lemonlabs.uri.Url

val relativeUrl = Url.parse("/example?a=b")
relativeUrl.withScheme("http").withHost("www.example.com") // This is http://www.example.com/example?a=b

Redacting URLs

It is possible to print out redacted URLs to logs with sensitive information either removed or replaced with a placeholder

Replacing with a placeholder:

import io.lemonlabs.uri._
import io.lemonlabs.uri.redact._

val url = Url.parse("http://user:[email protected]?secret=123&last=yes")

// This returns http://xxx:[email protected]?secret=xxx&last=yes
url.toRedactedString(Redact.withPlaceholder("xxx").params("secret", "other").user().password())

Removing:

import io.lemonlabs.uri._
import io.lemonlabs.uri.redact._

val url = Url.parse("http://user:[email protected]?secret=123&other=true")

// This returns http://example.com
url.toRedactedString(Redact.byRemoving.allParams().userInfo())

Url Equality

By default scala-uri only considers Urls equal if query parameters are in the same order:

import io.lemonlabs.uri._

val urlOne = Url.parse("https://example.com?a=1&b=2")
val urlTwo = Url.parse("https://example.com?b=2&a=1")

urlOne == urlTwo // this is false

val urlThree = Url.parse("https://example.com?a=1&b=2")

urlOne == urlThree // this is true

For use-cases where query parameter order is not important, the equalsUnordered can be used

urlOne.equalsUnordered(urlTwo) // this is true

When using cats for equality testing, parameter order will also be considered by default

import cats.implicits._

urlOne === urlTwo   // this is false
urlOne === urlThree // this is true

With cats, query parameter order can be ignored for equality checks with the following import:

import io.lemonlabs.uri.Url.unordered._

urlOne === urlTwo   // this is true
urlOne === urlThree // this is true

Note: depending on the type you are comparing, you will need to import a different cats Eq instance. The following are available:

import io.lemonlabs.uri.Uri.unordered._
import io.lemonlabs.uri.Url.unordered._
import io.lemonlabs.uri.RelativeUrl.unordered._
import io.lemonlabs.uri.UrlWithAuthority.unordered._
import io.lemonlabs.uri.ProtocolRelativeUrl.unordered._
import io.lemonlabs.uri.AbsoluteUrl.unordered._
import io.lemonlabs.uri.UrlWithoutAuthority.unordered._
import io.lemonlabs.uri.SimpleUrlWithoutAuthority.unordered._
import io.lemonlabs.uri.QueryString.unordered._

Pattern Matching URIs

import io.lemonlabs.uri._

val uri: Uri = Uri.parse("...")
uri match {
    case Uri(path) => // Matches Urns and Urls
    case Urn(path) => // Matches Urns
    case Url(path, query, fragment) => // Matches Urls
    case RelativeUrl(path, query, fragment) => // Matches RelativeUrls
    case UrlWithAuthority(authority, path, query, fragment) => // Matches AbsoluteUrl and ProtocolRelativeUrl
    case AbsoluteUrl(scheme, authority, path, query, fragment) => // Matches AbsoluteUrl
    case ProtocolRelativeUrl(authority, path, query, fragment) => // Matches ProtocolRelativeUrl
    case UrlWithoutAuthority(scheme, path, query, fragment) => // Matches UrlWithoutAuthorityUrl
    case DataUrl(mediaType, base64, data) => // Matches DataUrl
    case ScpLikeUrl(user, host, path) => // Matches ScpLikeUrl
}

Exhaustive matching

In some cases scalac will be able to detect instances where not all cases are being matched. For example:

import io.lemonlabs.uri._

Uri.parse("/test") match {
  case u: Url => println(u.toString)
}

results in the following compiler warning, because Uri.parse can return Urns as well as Urls:

<console>:15: warning: match may not be exhaustive.
It would fail on the following input: Urn(_)

In this instance, using Url.parse instead of Uri.parse would fix this warning

Hosts

Parsing Hosts

You can parse a String representing the host part of a URI with Host.parse. The return type is Host with an underling implementation of DomainName, IpV4 or IpV6.

import io.lemonlabs.uri.Host

val host = Host.parse("lemonlabs.io")

Parsing IPs

import io.lemonlabs.uri.{IpV4, IpV6}

val ipv4 = IpV4.parse("13.32.214.142")
val ipv6 = IpV6.parse("[1:2:3:4:5:6:7:8]")

Matching Hosts

import io.lemonlabs.uri._

val host: Host = Host.parse("...")
host match {
    case Host(host) => // Matches DomainNames, IpV4s and IpV6s
    case DomainName(host) => // Matches DomainNames
    case ip: IpV4 => // Matches IpV4s
    case ip: IpV6 => // Matches IpV6s
}

Paths

Matching Paths

import io.lemonlabs.uri._

val path: Path = Path.parse("...")
path match {
    case Path(parts) => // Matches any path
    case AbsolutePath(parts) => // Matches any path starting with a slash
    case RootlessPath(parts) => // Matches any path that *doesn't* start with a slash

    case PathParts("a", "b", "c") => // Matches "/a/b/c" and "a/b/c"
    case PathParts("a", "b", _*) => // Matches any path starting with "/a/b" or "a/b"

    case EmptyPath() => // Matches ""
    case PathParts() => // Matches "" and "/"

    case UrnPath("nid", "nss") => // Matches a URN Path "nid:nss"
}

URL Percent Encoding

By Default, scala-uri will URL percent encode paths and query string parameters. To prevent this, you can call the uri.toStringRaw method:

import io.lemonlabs.uri.Url

val uri = Url.parse("http://example.com/path with space?param=üri")

uri.toString // This is: http://example.com/path%20with%20space?param=%C3%BCri

uri.toStringRaw // This is: http://example.com/path with space?param=üri

The characters that scala-uri will percent encode by default can be found here. You can modify which characters are percent encoded like so:

Only percent encode the hash character:

import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.encoding._

implicit val config: UriConfig = UriConfig(encoder = percentEncode('#'))

Percent encode all the default chars, except the plus character:

import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.encoding._

implicit val config: UriConfig = UriConfig(encoder = percentEncode -- '+')

Encode all the default chars, and also encode the letters a and b:

import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.encoding._

implicit val config: UriConfig = UriConfig(encoder = percentEncode ++ ('a', 'b'))

Encoding spaces as pluses

The default behaviour with scala-uri, is to encode spaces as + in the querystring and as %20 elsewhere in the URL.

If you instead wish spaces to be encoded as %20 in the query, then simply add the following implicit val to your code:

import io.lemonlabs.uri.Url
import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.encoding._
import io.lemonlabs.uri.encoding.PercentEncoder._

implicit val config: UriConfig = UriConfig.default.copy(queryEncoder = PercentEncoder())

val uri = Url.parse("http://theon.github.com?test=uri with space")
uri.toString // This is http://theon.github.com?test=uri%20with%20space

The default behaviour with scala-uri, is to decode + in query string parameters to spaces and to leave it as a literal + elsewhere in the URL.

If you instead wish + to be left as + in the query, then simply add the following implicit val to your code:

import io.lemonlabs.uri.Url
import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.decoding._

implicit val config: UriConfig = UriConfig.default.copy(queryDecoder = PercentDecoder)

val uri = Url.parse("http://theon.github.com?test=uri+with+plus")
uri.query.param("test") // This is Some("uri+with+plus")

Custom encoding

If you would like to do some custom encoding for specific characters, you can use the encodeCharAs encoder.

import io.lemonlabs.uri.Url
import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.encoding._

implicit val config: UriConfig = UriConfig(encoder = percentEncode + encodeCharAs(' ', "_"))

val uri = Url.parse("http://theon.github.com/uri with space")
uri.toString // This is http://theon.github.com/uri_with_space

URL Percent Decoding

By Default, scala-uri will URL percent decode paths and query string parameters during parsing:

import io.lemonlabs.uri.Url

val uri = Url.parse("http://example.com/i-have-%25been%25-percent-encoded")

uri.toString // This is: http://example.com/i-have-%25been%25-percent-encoded
uri.toStringRaw // This is: http://example.com/i-have-%been%-percent-encoded

To prevent this, you can bring the following implicit into scope:

import io.lemonlabs.uri.Url
import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.decoding.NoopDecoder

implicit val c: UriConfig = UriConfig(decoder = NoopDecoder)

val uri = Url.parse("http://example.com/i-havent-%been%-percent-encoded")

uri.toString // This is: http://example.com/i-havent-%25been%25-percent-encoded
uri.toStringRaw // This is: http://example.com/i-havent-%been%-percent-encoded

Invalid Percent Encoding

If your Uri contains invalid percent encoding, by default scala-uri will throw a UriDecodeException:

Url.parse("/?x=%3") // This throws a UriDecodeException

You can configure scala-uri to instead ignore invalid percent encoding and only percent decode correctly percent encoded values like so:

import io.lemonlabs.uri.Url
import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.decoding.PercentDecoder

implicit val c: UriConfig = UriConfig(
  decoder = PercentDecoder(ignoreInvalidPercentEncoding = true)
)
val uri = Url.parse("/?x=%3")
uri.toString // This is /?x=%253
uri.toStringRaw // This is /?x=%3

Replacing Query String Parameters

If you wish to replace all existing query string parameters with a given name, you can use the Url.replaceParams() method:

import io.lemonlabs.uri.Url

val uri = Url.parse("http://example.com/path?param=1")
val newUri = uri.replaceParams("param", "2")

newUri.toString // This is: http://example.com/path?param=2

Removing Query String Parameters

If you wish to remove all existing query string parameters with a given name, you can use the uri.removeParams() method:

import io.lemonlabs.uri.Url

val uri = Url.parse("http://example.com/path?param=1&param2=2")
val newUri = uri.removeParams("param")

newUri.toString // This is: http://example.com/path?param2=2

Omitting Query Parameters with value `None`

scala-uri has support for not rendering query parameters that have a value of None. Set renderQuery = ExcludeNones in your UriConfig and make it visible in the scope where you parse/create your Url

import io.lemonlabs.uri.Url
import io.lemonlabs.uri.config._

implicit val config: UriConfig = UriConfig(renderQuery = ExcludeNones)

val url = Url.parse("http://github.com/lemonlabsuk").addParams("a" -> Some("some"), "b" -> None)
url.toString // This is http://github.com/lemonlabsuk?a=some

Get query string parameters

To get the query string parameters as a Map[String,Seq[String]] you can do the following:

import io.lemonlabs.uri.Url

val uri = Url.parse("http://example.com/path?a=b&a=c&d=e")
uri.query.paramMap // This is: Map("a" -> Vector("b", "c"), "d" -> Vector("e"))

User Information

scala-uri supports user information (username and password) encoded in URLs.

Parsing URLs with user information:

import io.lemonlabs.uri.Url

val url = Url.parse("http://user:[email protected]")
url.user // This is Some("user")
url.password // This is Some("pass")

Modifying user information:

import io.lemonlabs.uri.AbsoluteUrl

val url = AbsoluteUrl.parse("http://host.com")
url.withUser("jack") // URL is now http://[email protected]

import io.lemonlabs.uri.AbsoluteUrl

val url = AbsoluteUrl.parse("http://user:[email protected]")
url.withPassword("secret") // URL is now http://user:[email protected]

Note: that using clear text passwords in URLs is ill advised

Protocol Relative URLs

Protocol Relative URLs are supported in scala-uri. A Uri object with a protocol of None, but a host of Some(x) will be considered a protocol relative URL.

import io.lemonlabs.uri.Url

val uri = Url.parse("//example.com/path") // Return type is Url
uri.schemeOption // This is: None
uri.hostOption // This is: Some("example.com")

Use ProtocolRelativeUrl.parse if you know your URL will always be Protocol Relative:

import io.lemonlabs.uri.ProtocolRelativeUrl

val uri = ProtocolRelativeUrl.parse("//example.com/path") // Return type is ProtocolRelativeUrl
uri.schemeOption // This is: None
uri.host // This is: "example.com"

Character Sets

By default scala-uri uses UTF-8 charset encoding:

import io.lemonlabs.uri.Url

val uri = Url.parse("http://theon.github.com/uris-in-scala.html?chinese=网址")
uri.toString // This is http://theon.github.com/uris-in-scala.html?chinese=%E7%BD%91%E5%9D%80

This can be changed like so:

import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.Url

implicit val conf: UriConfig = UriConfig(charset = "GB2312")

val uri = Url.parse("http://theon.github.com/uris-in-scala.html?chinese=网址")
uri.toString // This is http://theon.github.com/uris-in-scala.html?chinese=%CD%F8%D6%B7

Subdomains

import io.lemonlabs.uri.Url

// This returns Some("www")
Url.parse("http://www.example.com/blah").subdomain

// This returns Some("a.b.c")
Url.parse("http://a.b.c.example.com/blah").subdomain

// This returns None
Url.parse("http://example.com/blah").subdomain

// This returns Vector("a", "a.b", "a.b.c", "a.b.c.example")
Url.parse("http://a.b.c.example.com/blah").subdomains

// This returns Some("a")
Url.parse("http://a.b.c.example.com/blah").shortestSubdomain

// This returns Some("a.b.c.example")
Url.parse("http://a.b.c.example.com/blah").longestSubdomain

These methods return None or Vector.empty for URLs without a Host (e.g. Relative URLs)

Apex Domains

The method apexDomain returns the apex domain for the URL (e.g. example.com for http://www.example.com/path)

import io.lemonlabs.uri.Url

val uri = Url.parse("http://www.google.co.uk/blah")
uri.apexDomain // This returns Some("google.co.uk")

Public Suffixes

scala-uri uses the list of public suffixes from publicsuffix.org to allow you to identify the TLD of your absolute URIs.

The publicSuffix method returns the longest public suffix from your URI

import io.lemonlabs.uri.Url

val uri = Url.parse("http://www.google.co.uk/blah")
uri.publicSuffix // This returns Some("co.uk")

The publicSuffixes method returns all the public suffixes from your URI

import io.lemonlabs.uri.Url

val uri = Url.parse("http://www.google.co.uk/blah")
uri.publicSuffixes // This returns Vector("co.uk", "uk")

These methods return None and Vector.empty, respectively for URLs without a Host (e.g. Relative URLs)

Punycode

See RFC 3490

import io.lemonlabs.uri.Url

val url = Url.parse("https://はじめよう.みんな/howto.html")
url.toStringPunycode // This returns "https://xn--p8j9a0d9c9a.xn--q9jyb4c/howto.html"

mailto

Mailto URLs are best parsed with UrlWithoutAuthority.parse, but can also be parsed with Url.parse

import io.lemonlabs.uri.UrlWithoutAuthority

val mailto = UrlWithoutAuthority.parse("mailto:[email protected]?subject=Hello")
mailto.scheme // This is Some(mailto")
mailto.path // This is "[email protected]"
mailto.query.param("subject") // This is Some("Hello")

Data URLs

Data URLs are defined in RFC2397

Base64 encoded data URLs

import java.io.ByteArrayInputStream
import io.lemonlabs.uri.DataUrl
import javax.imageio.ImageIO

// A data URL containing a PNG image of a red dot
val dataUrl = DataUrl.parse("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==")

dataUrl.scheme // This is "data"
dataUrl.mediaType.value // This is "image/png"
dataUrl.base64 // This is true

// Convert the image data to a java.awt.image.BufferedImage
val image = ImageIO.read(new ByteArrayInputStream(dataUrl.data))

Percent encoded data URLs

import io.lemonlabs.uri.DataUrl

val dataUrl = DataUrl.parse("data:text/plain;charset=UTF-8;page=21,the%20data:1234,5678")

dataUrl.mediaType.value // This is text/plain
dataUrl.mediaType.charset // This is UTF-8
dataUrl.mediaType.parameters // This is Vector("charset" -> "UTF-8", "page" -> "21")
dataUrl.base64 // This is false

dataUrl.dataAsString // This is "the data:1234,5678"

git/scp style URLs

git/scp style URLs can be parsed like so:

import io.lemonlabs.uri.ScpLikeUrl

val url = ScpLikeUrl.parse("[email protected]:lemonlabsuk/scala-uri.git")
url.user // This is Some("git")
url.host.toString // This is "github.com"
url.path.toString // This is "lemonlabsuk/scala-uri.git"

Note that ScpLikeUrl.parse, should only be used for git URLs with scp-like syntax (with a : between the host and path). For all other git URLs, AbsoluteUrl.parse or Url.parse should be used:

import io.lemonlabs.uri.AbsoluteUrl

val gitUrl    = AbsoluteUrl.parse("git://github.com/lemonlabsuk/scala-uri.git")
val gitSshUrl = AbsoluteUrl.parse("git+ssh://[email protected]/lemonlabsuk/scala-uri.git")
val sshUrl    = AbsoluteUrl.parse("ssh://[email protected]/lemonlabsuk/scala-uri.git")
val httpsUrl  = AbsoluteUrl.parse("https://github.com/lemonlabsuk/scala-uri.git")

Typesafe URL builder DSL

The version of DSL which relies on the types to render urls providing better control over the way values would be translated to url parts.

It is possible to use arbitrary types as parts of the url:

Query Strings

import io.lemonlabs.uri.typesafe._
import io.lemonlabs.uri.typesafe.dsl._

final case class Foo(a: Int, b: String)

object Foo {
  implicit val traversableParams: TraversableParams[Foo] = TraversableParams.product
}

val uri = "http://theon.github.com/scala-uri" addParams Foo(a = 1, b = "bar")
uri.toString //This is: http://theon.github.com/scala-uri?a=1&b=bar

Query String Values

import io.lemonlabs.uri.typesafe._
import io.lemonlabs.uri.typesafe.dsl._

sealed trait Bar {
  def name: String
}

case object A extends Bar {
  val name: String = "A"
}

case object B extends Bar {
  val name: String = "B"
}

object Bar {
  implicit val queryValue: QueryValue[Bar] = QueryValue.derive[Bar].by(_.name)
}

val uri = "http://theon.github.com/scala-uri" ? ("foo" -> A)
uri.toString //This is: http://theon.github.com/scala-uri?foo=A

Path Parts

import io.lemonlabs.uri.typesafe._
import io.lemonlabs.uri.typesafe.dsl._

final case class Foo(a: String, b: Int)

object Foo {
  implicit val pathPart: PathPart[Foo] = (foo: Foo) => s"${foo.a}/${foo.b}"
}


val uri = "http://theon.github.com" / "scala-uri" / Foo(a = "user", b = 1)
uri.toString //This is: http://theon.github.com/scala-uri/user/1

Fragments

import io.lemonlabs.uri.typesafe._
import io.lemonlabs.uri.typesafe.dsl._

final case class Foo(a: String, b: Int)
object Foo {
  implicit val fragment: Fragment[Foo] = (foo: Foo) => Some(s"${foo.a}-${foo.b}")
}

val uri5 = "http://theon.github.com/scala-uri" `#` Foo(a = "user", b = 1)
uri5.toString //This is: http://theon.github.com/scala-uri#user-1

Scala.js support

See scala-uri-scalajs-example for usage

Cats Support

scala-uri provides type class instances of cats.Eq, cats.Show and cats.Order for: Uri , Url, RelativeUrl, UrlWithAuthority, ProtocolRelativeUrl, AbsoluteUrl, UrlWithoutAuthority, SimpleUrlWithoutAuthority, DataUrl, ScpLikeUrl, Urn, Authority, UserInfo, Host, DomainName, IpV4, IpV6, MediaType, Path, UrlPath, AbsoluteOrEmptyPath, RootlessPath, AbsolutePath, UrnPath, QueryString

The type class instances exist in the companion objects for these types.

Including scala-uri your project

scala-uri 4.x.x is currently built with support for Scala 3, Scala 2.13.x, Scala 2.12.x and Scala.js 1.1.0+ scala-uri 3.x.x is currently built with support for Scala 2.13.x, Scala 2.12.x and Scala.js 1.1.0+

For 2.11.x support use scala-uri 1.4.10 from branch 1.4.x
For 2.10.x support use scala-uri 0.4.17 from branch 0.4.x
For 2.9.x support use scala-uri 0.3.6 from branch 0.3.x
For Scala.js 1.x.x support, use scala-uri 4.0.0
For Scala.js 0.6.x support, use scala-uri 2.2.3

Release builds are available in maven central. For SBT users just add the following dependency:

"io.lemonlabs" %% "scala-uri" % "4.0.4"

For maven users you should use (for 2.13.x):

<dependency>
    <groupId>io.lemonlabs</groupId>
    <artifactId>scala-uri_2.13</artifactId>
    <version>4.0.4</version>
</dependency>

Contributions

Contributions to scala-uri are always welcome. Check out the Contributing Guidelines

Migration guides

3.x.x to 4.x.x

Scala 3 support has been added. Scala 2.13 and 2.12 support remain
Binary Incompatibility: 4765b4e removed a single UriConfig.copy() overload. This overload existed only to maintain binary compatibility with an older version of scala-uri Use the remaining copy() method on UriConfig instead
Binary Incompatibility: 4765b4e removed a single UriConfig overloaded constructor. This overload existed only to maintain binary compatibility with an older version of scala-uri Use the remaining constructor or apply method remaining on UriConfig instead

2.x.x to 3.x.x

Backwards Incompatible: The space character is now encoded to + instead of %20 in query string parameters by default.
Backwards Incompatible: The + method in io.lemonlabs.uri.encoding.UriEncoder, now chains encoders in the opposite order to be more intuitive. E.g. a + b will encode with encoder a first, followed by encoder b
Binary Incompatibility: The following deprecated classes have now been removed:
- io.lemonlabs.uri.inet.PublicSuffixTrie
- io.lemonlabs.uri.inet.Trie
- io.lemonlabs.uri.dsl.*
Binary Incompatibility: Url.addParam("key", "value") and Url.addParam("key" -> "value") now has a return type Self rather than Url. E.g. AbsoluteUrl.addParam now has a return type of AbsoluteUrl and RelativeUrl.addParam has a return type of RelativeUrl

1.x.x to 2.x.x

scala-uri no longer depends on a JSON library.
Binary Incompatibility: The case class UrlWithoutAuthority has been renamed SimpleUrlWithoutAuthority. There is now a trait called UrlWithoutAuthority. This trait has a companion object with apply, unapply and parse methods, so it mostly can be used in the same way as the previous case class.
Binary Incompatibility: Parsing a Data URL will now return an instance of DataUrl rather than UrlWithoutAuthority
Binary Incompatibility: UserInfo.user is now of type String rather than Option[String]
Binary Incompatibility: Authority.userInfo is now of type Option[UserInfo]
Binary Incompatibility: UserInfo.empty method removed
Binary Incompatibility: QueryString.fromPairOptions removed. Use QueryString.fromPairs instead.
Binary Incompatibility: Url.withQueryStringOptionValues removed. Use withQueryString instead.
Binary Incompatibility: Url.addParamsOptionValues and QueryString.addParamsOptionValues have been renames to addParams
TypesafeUrlDsl
- Binary Incompatibility: withParams[A: TraversableParams](params: A) renamed to addParams
- /(PathPart) no longer splits the part by slash. If you want to add multiple path parts use /(TraversablePathParts) instead
Type Classes
- Binary Incompatibility: Fragment[A].fragment returns Option[String] rather than String
- Binary Incompatibility: Url.withFragment now takes argument of type T: Fragment rather than String and Option[String] Type Class instances are provided the method can be used with String and Option[String] values just as before
- Binary Incompatibility: Url.addPathParts and Url.addPathPart now takes arguments of type P: TraversablePathParts or P: PathPart rather than Iterable[String] or String Type Class instances are provided the methods can be used with String and Iterable[String] values just as before
- Binary Incompatibility: Url.withQueryString, Url.addParam, Url.addParams, Url.replaceParams, Url.removeParams, Url.mapQuery, Url.flatMapQuery, Url.collectQuery, Url.mapQueryNames and Url.mapQueryValues now takes argument of type KV: QueryKeyValue, K: QueryKey or V: QueryValue rather than (String, String) or String Type Class instances are provided the methods can be used with (String, String) or String values just as before
The URL builder DSL has been deprecated in favour of the Typesafe URL builder DSL
Authority.parse no longer expects it's string argument to start with //, as this is not part of the Authority, it is a delimiter. See RFC 3986
UserInfo.parse no longer expects it's string argument to end with a @, as this is not part of the UserInfo, it is a delimiter. See RFC 3986
QueryString.toString no longer returns a leading ?, as this is not part of the query string, it is a delimiter. See RFC 3986
Forward slashes in paths are now percent encoded by default. This means Url.parse("/%2F/").toString returns "/%2F/" rather than /// in previous versions To return to the previous behavior, you can bring a UriConfig like so into scope
```
 import io.lemonlabs.uri.encoding.PercentEncoder._
 implicit val c = UriConfig.default.copy(pathEncoder = PercentEncoder(PATH_CHARS_TO_ENCODE - '/'))
```

1.x.x to 1.5.x

scala 2.11 support dropped, please upgrade to 2.12 or 2.13

0.5.x to 1.x.x

Thanks to @evanbennett. 1.x.x is inspired by his fork here and discussion here.

Package change from com.netaporter.uri to io.lemonlabs.uri
The single Uri case class has now been replaced with a class hierarchy. Use the most specific class in this hierarchy that fits your use case
Uri used to be a case class, but the replacements Uri and Url are now traits. This means they no longer have a copy method. Use the with methods instead (e.g. withHost, withPath etc)
host method on Url now has return type Host rather than String. You may have to change url.host to url.host.toString
path method on Url now has return type Path rather than String. You may have to change url.path to url.path.toString
Changed parameter value type from Any to String in methods addParam, addParams, replaceParams. Please now call .toString before passing non String types to these methods
Changed parameter value type from Option[Any] to Option[String] in method replaceAll. Please now call .toString before passing non String types to this method
Query string parameters with a value of None will now be rendered with no equals sign by default (e.g. ?param). Previously some methods (such as ?, &, \?, addParam and addParams) would not render parameters with a value of None at all. In 1.x.x, this behaviour can be achieved by using the renderQuery config option.
In most cases Url.parse should be used instead of Uri.parse. See all parse methods here
scheme is now called schemeOption on Uri. If you have an instance of AbsoluteUrl or ProtocolRelativeUrl there is still scheme method but it returns String rather than Option[String]
protocol method has been removed from Uri. Use schemeOption instead
Type changed from Seq to Vector for:
- subdomains, publicSuffixes, params return type
- removeAll and removeParams argument types
- params field in QueryString
- paramMap and pathParts fields in Uri, now Url
Methods addParam and addParams that took Option arguments are now called addParamOptionValue and addParamsOptionValues
Method replaceAllParams has been replaced with withQueryString or withQueryStringOptionValues
Method removeAllParams has been replaced with withQueryString(QueryString.empty)
Method subdomain has been removed from the Scala.js version. The implementation was incorrect and did not match the JVM version of subdomain. Once public suffixes are supported for the Scala.js version, a correct implementation of subdomain can be added
Implicit UriConfigs now need to be where your Uris are parsed/constructed, rather than where they are rendered
Method hostParts has been removed from Uri. This method predated publicSuffix and subdomain which are more useful methods for pulling apart a host
Field pathStartsWithSlash removed from Uri. This was only intended to be used internally. You can now instead check if Uri.path is an instance of AbsolutePath to determine if the path will start with slash

0.4.x to 0.5.x

Matrix parameters have been removed. If you still need this, raise an issue
scala 2.10 support dropped, please upgrade to 2.11 or 2.12 to use scala-uri 0.5.x
Scala.js support added

0.3.x to 0.4.x

Package changes / import changes
All code moved from com.github.theon package to com.netaporter package
scala-uri has been organised into the following packages: encoding, decoding, config and dsl. You will need to update import statments.
Name changes
PermissiveDecoder renamed to PermissivePercentDecoder
QueryString and MatrixParams constructor argument parameters shortened to params
Uri.parseUri renamed to Uri.parse
protocol constructor arg in Uri renamed to scheme
Querystring renamed to QueryString
Query String constructor argument parameters changed type from Map[String, List[String]] to Seq[(String,String)]
Uri constructor argument pathParts changed type from List to Vector
Uri method to add query string parameters renamed from params to addParams. Same with matrixParams -> addMatrixParams
PercentEncoderDefaults object renamed to PercentEncoder companion object.
Copy methods user/password/port/host/scheme now all prefixed with with, e.g. withHost
New UriConfig case class used to specify encoders, decoders and charset to be used. See examples in Custom encoding, URL Percent Decoding and Character Sets

License

scala-uri is open source software released under the Apache 2 License.

Name		Name	Last commit message	Last commit date
Latest commit History 1,227 Commits
.github		.github
docs		docs
js/src/main/scala/io/lemonlabs/uri/inet		js/src/main/scala/io/lemonlabs/uri/inet
jvm/src		jvm/src
project		project
shared/src		shared/src
.gitignore		.gitignore
.gitpod.Dockerfile		.gitpod.Dockerfile
.gitpod.yml		.gitpod.yml
.jvmopts		.jvmopts
.mergify.yml		.mergify.yml
.nvmrc		.nvmrc
.scalafmt.conf		.scalafmt.conf
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
build.sbt		build.sbt
bumpversion.sh		bumpversion.sh
codecov.yml		codecov.yml
publish.sh		publish.sh
pull_request_template.md		pull_request_template.md
version.sbt		version.sbt

License

ActianCorp/scala-uri

Folders and files

Latest commit

History

Repository files navigation

scala-uri

Migration Guides

Parsing

Parse a URL

Parse a URN

Parse a URI

Building URLs

Transforming URLs

mapQuery

filterQuery

collectQuery

Convert an Absolute URL to a Relative URL

Convert a Relative URL to an Absolute URL

Redacting URLs

Url Equality

Pattern Matching URIs

Exhaustive matching

Hosts

Parsing Hosts

Parsing IPs

Matching Hosts

Paths

Matching Paths

URL Percent Encoding

Encoding spaces as pluses

Custom encoding

URL Percent Decoding

Invalid Percent Encoding

Replacing Query String Parameters

Removing Query String Parameters

Omitting Query Parameters with value None

Get query string parameters

User Information

Protocol Relative URLs

Character Sets

Subdomains

Apex Domains

Public Suffixes

Punycode

mailto

Data URLs

Base64 encoded data URLs

Percent encoded data URLs

git/scp style URLs

Typesafe URL builder DSL

Query Strings

Query String Values

Path Parts

Fragments

Scala.js support

Cats Support

Including scala-uri your project

Contributions

Migration guides

3.x.x to 4.x.x

2.x.x to 3.x.x

1.x.x to 2.x.x

1.x.x to 1.5.x

0.5.x to 1.x.x

0.4.x to 0.5.x

0.3.x to 0.4.x

License

About

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

Packages 0

Languages

Omitting Query Parameters with value `None`

Packages