Improve documentation (#2385)

README.md

@@ -11,13 +11,31 @@
 
 ## Welcome!
 
-[sttp client](https://github.com/softwaremill/sttp) is an open-source library which provides a clean, programmer-friendly API to describe HTTP
-requests and how to handle responses. Requests are sent using one of the backends, which wrap lower-level Scala or Java HTTP client implementations. The backends can integrate with a variety of Scala stacks, providing both synchronous and asynchronous, procedural and functional interfaces.
-
-Backend implementations include the HTTP client that is shipped with Java, as well as ones based on [akka-http](https://doc.akka.io/docs/akka-http/current/scala/http/), [http4s](https://http4s.org), [OkHttp](http://square.github.io/okhttp/). They integrate with [Akka](https://akka.io), [Monix](https://monix.io), [fs2](https://github.com/functional-streams-for-scala/fs2), [cats-effect](https://github.com/typelevel/cats-effect), [scalaz](https://github.com/scalaz/scalaz) and [ZIO](https://github.com/zio/zio). Supported Scala versions include 2.12, 2.13 and 3, Scala.JS and Scala Native; supported Java versions include 11+.
+sttp client is an open-source HTTP client for Scala, supporting various approaches to writing Scala code: synchronous (direct-style), `Future`-based, and using functional effect systems (cats-effect, ZIO, Monix, Kyo, scalaz).
+
+The library is available for Scala 2.12, 2.13 and 3. Supported platforms are the JVM (Java 11+), Scala.JS and Scala Native.
 
 Here's a quick example of sttp client in action, runnable using [scala-cli](https://scala-cli.virtuslab.org):
-
+
+```scala
+//> using dep com.softwaremill.sttp.client4::core:4.0.0-M22
+
+import sttp.client4.quick.*
+
+@main def run(): Unit =
+  println(quickRequest.get(uri"http://httpbin.org/ip").send())
+```
+
+sttp client addresses common HTTP client use cases, such as interacting with JSON APIs (with automatic serialization of request bodies and deserialization of response bodies), uploading and downloading files, submitting form data, handling multipart requests, and working with WebSockets.
+
+The driving principle of sttp client's design is to provide a clean, programmer-friendly API to describe HTTP requests, along with response handling. This ensures that resources, such as HTTP connections, are used safely, also in the presence of errors.
+
+sttp client integrates with a number of lower-level Scala and Java HTTP client implementations through backends (using Java's `HttpClient`, Akka HTTP, Pekko HTTP, http4s, OkHttp, Armeria), offering a wide range of choices when it comes to protocol support, connectivity settings and programming stack compatibility. 
+
+Additionally, sttp client seamlessly integrates with popular libraries for JSON handling (e.g., circe, uPickle, jsoniter, json4s, play-json, ZIO Json), logging, metrics, and tracing (e.g., slf4j, scribe, OpenTelemetry, Prometheus). It also supports streaming libraries (e.g., fs2, ZIO Streams, Akka Streams, Pekko Streams) and provides tools for testing HTTP interactions.
+
+Some more features: URI interpolation, a self-managed backend, and type-safe HTTP error/success representation, are demonstrated by the below example:
+
 ```scala
 //> using dep com.softwaremill.sttp.client4::core:4.0.0-M22
 
@@ -29,18 +47,21 @@ import sttp.client4.*
 
   // the `query` parameter is automatically url-encoded
   // `sort` is removed, as the value is not defined
-  val request = basicRequest.get(uri"https://api.github.com/search/repositories?q=$query&sort=$sort")
+  val request = basicRequest.get(
+    uri"https://api.github.com/search/repositories?q=$query&sort=$sort")
 
   val backend = DefaultSyncBackend()
   val response = request.send(backend)
 
   // response.header(...): Option[String]
   println(response.header("Content-Length")) 
 
-  // response.body: by default read into an Either[String, String] to indicate failure or success 
+  // response.body: read into an Either[String, String] to indicate failure or success 
   println(response.body)
 ```
 
+But that's just a small glimpse of sttp client's features! For more examples, see the [usage examples](https://sttp.softwaremill.com/en/latest/examples.html). 
+
 ## Documentation
 
 sttp (v4) documentation is available at [sttp.softwaremill.com/en/latest](https://sttp.softwaremill.com/en/latest).
@@ -60,7 +81,7 @@ If you are using [scala-cli](https://scala-cli.virtuslab.org), you can quickly s
 
 ```
 //> using dep "com.softwaremill.sttp.client4::core:4.0.0-M22"
-import sttp.client4.quick._
+import sttp.client4.quick.*
 quickRequest.get(uri"http://httpbin.org/ip").send()
 ```
 
@@ -72,7 +93,7 @@ Similarly, using [Ammonite](http://ammonite.io):
 
 ```scala
 import $ivy.`com.softwaremill.sttp.client4::core:4.0.0-M22`
-import sttp.client4.quick._
+import sttp.client4.quick.*
 quickRequest.get(uri"http://httpbin.org/ip").send()
 ```
 
@@ -87,7 +108,7 @@ Add the following dependency:
 Then, import:
 
 ```scala
-import sttp.client4._
+import sttp.client4.*
 ```
 
 Type `basicRequest.` and see where your IDE’s auto-complete gets you!
@@ -97,10 +118,11 @@ Type `basicRequest.` and see where your IDE’s auto-complete gets you!
 sttp is a family of Scala HTTP-related projects, and currently includes:
 
 * sttp client: this project
-* [sttp tapir](https://github.com/softwaremill/tapir): Typed API descRiptions
+* [sttp tapir](https://github.com/softwaremill/tapir): rapid development of self-documenting APIs
 * [sttp model](https://github.com/softwaremill/sttp-model): simple HTTP model classes (used by client & tapir)
 * [sttp shared](https://github.com/softwaremill/sttp-shared): shared web socket, FP abstractions, capabilities and streaming code.
 * [sttp apispec](https://github.com/softwaremill/sttp-apispec): OpenAPI, AsyncAPI and JSON Schema models.
+* [sttp openai](https://github.com/softwaremill/sttp-openai): Scala client wrapper for OpenAI and OpenAI-compatible APIs. Use the power of ChatGPT inside your code!
 
 ## Contributing
 
@@ -165,4 +187,4 @@ We offer commercial support for sttp and related technologies, as well as develo
 
 ## Copyright
 
-Copyright (C) 2017-2024 SoftwareMill [https://softwaremill.com](https://softwaremill.com).
+Copyright (C) 2017-2025 SoftwareMill [https://softwaremill.com](https://softwaremill.com).

build.sbt

@@ -36,18 +36,8 @@ val compileScoped =
 val testScoped =
   inputKey[Unit](s"Run tests in the given scope. Usage: testScoped [scala version] [platform]. $scopesDescription")
 
-val commonSettings = commonSmlBuildSettings ++ ossPublishSettings ++ Seq(
+val commonSettings = commonSmlBuildSettings ++ Seq(
   organization := "com.softwaremill.sttp.client4",
-  updateDocs := Def.taskDyn {
-    val files1 = UpdateVersionInDocs(sLog.value, organization.value, version.value, List(file("README.md")))
-    Def.task {
-      (docs.jvm(scala3) / mdoc).toTask("").value
-      // Generating the list only after mdoc is done (as it overrides what's in generated_doc)
-      // For the root project the sourceDirectory points to src, so ../ will point to the root directory of the project
-      GenerateListOfExamples(sLog.value, sourceDirectory.value.getParentFile)
-      files1 ++ Seq(file("generated-docs/out"))
-    }
-  }.value,
   ideSkipProject := (scalaVersion.value != ideScalaVersion)
     || thisProjectRef.value.project.contains("JS") || thisProjectRef.value.project.contains("Native"),
   bspEnabled := !ideSkipProject.value,
@@ -274,7 +264,8 @@ def filterByVersionAndPlatform(scalaVersionFilter: String, platformFilter: Strin
 }
 
 lazy val rootProject = (project in file("."))
-  .settings(commonSettings: _*)
+  .settings(commonSettings)
+  .settings(ossPublishSettings)
   .settings(
     publish / skip := true,
     name := "sttp",
@@ -287,7 +278,17 @@ lazy val rootProject = (project in file("."))
       Def.taskDyn((Test / test).all(filterByVersionAndPlatform(args.head, args(1))))
     }.evaluated,
     ideSkipProject := false,
-    scalaVersion := scala2_13
+    scalaVersion := scala2_13,
+    updateDocs := Def.taskDyn {
+      val files1 = UpdateVersionInDocs(sLog.value, organization.value, version.value, List(file("README.md")))
+      Def.task {
+        (docs.jvm(scala3) / mdoc).toTask("").value
+        // Generating the list only after mdoc is done (as it overrides what's in generated_doc)
+        // For the root project the sourceDirectory points to src, so ../ will point to the root directory of the project
+        GenerateListOfExamples(sLog.value, sourceDirectory.value.getParentFile)
+        files1 ++ Seq(file("generated-docs/out"))
+      }
+    }.value
   )
   .aggregate(allAggregates: _*)
 
@@ -972,6 +973,9 @@ lazy val examples = (projectMatrix in file("examples"))
     libraryDependencies ++= Seq(
       "io.circe" %% "circe-generic" % circeVersion,
       "org.json4s" %% "json4s-native" % json4sVersion,
+      "com.github.plokhotnyuk.jsoniter-scala" %%% "jsoniter-scala-macros" % jsoniterVersion,
+      "io.github.resilience4j" % "resilience4j-circuitbreaker" % resilience4jVersion,
+      "io.github.resilience4j" % "resilience4j-ratelimiter" % resilience4jVersion,
       pekkoStreams,
       logback
     ),
@@ -986,6 +990,8 @@ lazy val examples = (projectMatrix in file("examples"))
     json4s,
     circe,
     upickle,
+    jsoniter,
+    zioJson,
     scribeBackend,
     slf4jBackend,
     ox
@@ -1018,8 +1024,6 @@ lazy val docs: ProjectMatrix = (projectMatrix in file("generated-docs")) // impo
       "io.circe" %% "circe-generic" % circeVersion,
       "com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-macros" % jsoniterVersion,
       "commons-io" % "commons-io" % "2.18.0",
-      "io.github.resilience4j" % "resilience4j-circuitbreaker" % resilience4jVersion,
-      "io.github.resilience4j" % "resilience4j-ratelimiter" % resilience4jVersion,
       "io.jaegertracing" % "jaeger-client" % jaegerClientVersion,
       "io.opentracing.brave" % "brave-opentracing" % braveOpentracingVersion,
       "io.zipkin.reporter2" % "zipkin-sender-okhttp3" % zipkinSenderOkHttpVersion,

core/src/main/scala/sttp/client4/RetryWhen.scala

@@ -15,6 +15,14 @@ object RetryWhen {
       case m: MultipartBody[_] => m.parts.forall(p => isBodyRetryable(p.body))
     }
 
+  /** The default implementation of a predicate, checking if a request can be retried.
+    *
+    * A request can be retried if:
+    *   - there was a connection error (hence, the request was never sent)
+    *   - the response was a server error (5xx, not 4xx - which is a client's error)
+    *   - the request's method was idempotent (e.g. GET, but not POST)
+    *   - the body can be sent again (e.g. not a stream)
+    */
   val Default: RetryWhen = {
     case (_, Left(_: SttpClientException.ConnectException)) => true
     case (_, Left(_))                                       => false

core/src/main/scala/sttp/client4/package.scala

@@ -1,5 +1,10 @@
 package sttp
 
 package object client4 extends SttpApi {
+
+  /** The type of a predicate that can be used to determine, if a request should be retries.
+    * @see
+    *   [[RetryWhen.Default]]
+    */
   type RetryWhen = (GenericRequest[_, _], Either[Throwable, Response[_]]) => Boolean
 }

core/src/main/scala/sttp/client4/testing/BackendStub.scala

@@ -19,7 +19,6 @@ import scala.concurrent.ExecutionContext
   * Predicates can match requests basing on the URI or headers. A [[ClassCastException]] might occur if for a given
   * request, a response is specified with the incorrect or inconvertible body type.
   */
-
 class BackendStub[F[_]](
     monad: MonadError[F],
     matchers: PartialFunction[GenericRequest[_, _], F[Response[_]]],

docs/backends/akka.md

@@ -69,11 +69,11 @@ val response: Future[Response[Either[String, Source[ByteString, Any]]]] =
     .send(backend)           
 ```
 
-The akka-http backend support both regular and streaming [websockets](../websockets.md).
+The akka-http backend support both regular and streaming [websockets](../other/websockets.md).
 
 ## Testing
 
-Apart from testing using [the stub](../testing.md), you can create a backend using any `HttpRequest => Future[HttpResponse]` function, or an akka-http `Route`.
+Apart from testing using [the stub](../testing/stub.md), you can create a backend using any `HttpRequest => Future[HttpResponse]` function, or an akka-http `Route`.
 
 That way, you can "mock" a server that the backend will talk to, without starting any actual server or making any HTTP calls.
 

docs/backends/catseffect.md

@@ -1,7 +1,7 @@
 # cats-effect backend
 
 The [Cats Effect](https://github.com/typelevel/cats-effect) backend is **asynchronous**. 
-It can be created for any type implementing the `cats.effect.Concurrent` typeclass, such as `cats.effect.IO`. 
+It can be created for any type implementing the `cats.effect.kernel.Async` typeclass, such as `cats.effect.IO`. 
 Sending a request is a non-blocking, lazily-evaluated operation and results in a wrapped response. 
 There's a transitive dependency on `cats-effect`. 
 
@@ -82,9 +82,10 @@ Creation of the backend can be done in two basic ways:
 Firstly, add the following dependency to your project:
 
 ```scala
-"com.softwaremill.sttp.client4" %% "armeria-backend-cats" % "@VERSION@" // for cats-effect 3.x
-// or
-"com.softwaremill.sttp.client4" %% "armeria-backend-cats-ce2" % "@VERSION@" // for cats-effect 2.x
+// for cats-effect 3.x
+"com.softwaremill.sttp.client4" %% "armeria-backend-cats" % "@VERSION@" 
+// or for cats-effect 2.x
+"com.softwaremill.sttp.client4" %% "armeria-backend-cats-ce2" % "@VERSION@" 
 ```
 
 create client:
@@ -126,8 +127,8 @@ val client = WebClient.builder("https://my-service.com")
 val backend = ArmeriaCatsBackend.usingClient[IO](client)
 ```
 
-```{eval-rst}
-.. note:: A WebClient could fail to follow redirects if the WebClient is created with a base URI and a redirect location is a different URI.
+```{note}
+A WebClient could fail to follow redirects if the WebClient is created with a base URI and a redirect location is a different URI.
 ```
 
 This backend is build on top of [Armeria](https://armeria.dev/docs/client-http).
@@ -140,4 +141,4 @@ This backend doesn't support non-blocking [streaming](../requests/streaming.md).
 
 ## Websockets
 
-The backend doesn't support [websockets](../websockets.md).
+The backend doesn't support [websockets](../other/websockets.md).

docs/backends/finagle.md

@@ -17,4 +17,4 @@ This backend depends on [finagle](https://twitter.github.io/finagle/), and offer
 
 Please note that: 
 
-* the backend does not support non-blocking [streaming](../requests/streaming.md) or [websockets](../websockets.md).
+* the backend does not support non-blocking [streaming](../requests/streaming.md) or [websockets](../other/websockets.md).

docs/backends/fs2.md

@@ -1,6 +1,6 @@
 # fs2 backend
 
-The [fs2](https://github.com/functional-streams-for-scala/fs2) backends are **asynchronous**. They can be created for any type implementing the `cats.effect.Async` typeclass, such as `cats.effect.IO`. Sending a request is a non-blocking, lazily-evaluated operation and results in a wrapped response. There's a transitive dependency on `cats-effect`. 
+The [fs2](https://github.com/functional-streams-for-scala/fs2) backends are **asynchronous**. They can be created for any type implementing the `cats.effect.kernel.Async` typeclass, such as `cats.effect.IO`. Sending a request is a non-blocking, lazily-evaluated operation and results in a wrapped response. There's a transitive dependency on `cats-effect`. 
 
 ## Using HttpClient
 
@@ -77,9 +77,10 @@ Host header override is supported in environments running Java 12 onwards, but i
 To use, add the following dependency to your project:
 
 ```scala
-"com.softwaremill.sttp.client4" %% "armeria-backend-fs2" % "@VERSION@" // for cats-effect 3.x & fs2 3.x
-// or
-"com.softwaremill.sttp.client4" %% "armeria-backend-fs2" % "@VERSION@" // for cats-effect 2.x & fs2 2.x
+// for cats-effect 3.x & fs2 3.x
+"com.softwaremill.sttp.client4" %% "armeria-backend-fs2" % "@VERSION@" 
+// or for cats-effect 2.x & fs2 2.x
+"com.softwaremill.sttp.client4" %% "armeria-backend-fs2" % "@VERSION@" 
 ```
 
 create client:
@@ -117,8 +118,8 @@ val client = WebClient.builder("https://my-service.com")
 val backend = ArmeriaFs2Backend.usingClient[IO](client, dispatcher)
 ```
 
-```{eval-rst}
-.. note:: A WebClient could fail to follow redirects if the WebClient is created with a base URI and a redirect location is a different URI.
+```{note}
+A WebClient could fail to follow redirects if the WebClient is created with a base URI and a redirect location is a different URI.
 ```
 
 This backend is built on top of [Armeria](https://armeria.dev/docs/client-http).
@@ -174,7 +175,7 @@ val effect = HttpClientFs2Backend.resource[IO]().use { backend =>
 
 ## Websockets
 
-The fs2 backends support both regular and streaming [websockets](../websockets.md).
+The fs2 backends support both regular and streaming [websockets](../other/websockets.md).
 
 ## Server-sent events
 

docs/backends/future.md

@@ -125,8 +125,8 @@ val client = WebClient.builder("https://my-service.com")
 val backend = ArmeriaFutureBackend.usingClient(client)
 ```
 
-```{eval-rst}
-.. note:: A WebClient could fail to follow redirects if the WebClient is created with a base URI and a redirect location is a different URI.
+```{note}
+A WebClient could fail to follow redirects if the WebClient is created with a base URI and a redirect location is a different URI.
 ```
 
 This backend is build on top of [Armeria](https://armeria.dev/docs/client-http) and doesn't support host header override.
@@ -141,4 +141,4 @@ Other backends don't support non-blocking [streaming](../requests/streaming.md).
 
 ## Websockets
 
-Some of the backends (see above) support regular and streaming [websockets](../websockets.md).
+Some of the backends (see above) support regular and streaming [websockets](../other/websockets.md).

docs/backends/http4s.md

@@ -3,9 +3,10 @@
 This backend is based on [http4s](https://http4s.org) (client) and is **asynchronous**. To use, add the following dependency to your project:
 
 ```scala
-"com.softwaremill.sttp.client4" %% "http4s-backend" % "@VERSION@" // for cats-effect 3.x & http4s 1.0.0-Mx
-// or
-"com.softwaremill.sttp.client4" %% "http4s-ce2-backend" % "@VERSION@" // for cats-effect 2.x & http4s 0.21.x
+// for cats-effect 3.x & http4s 1.0.0-Mx
+"com.softwaremill.sttp.client4" %% "http4s-backend" % "@VERSION@" 
+// or for cats-effect 2.x & http4s 0.21.x
+"com.softwaremill.sttp.client4" %% "http4s-ce2-backend" % "@VERSION@"
 ```
 
 The backend can be created in a couple of ways, e.g.:
@@ -37,4 +38,4 @@ Instead, all custom timeout configuration should be done by creating a `org.http
 
 The backend supports streaming using fs2. For usage details, see the documentation on [streaming using fs2](fs2.md).
 
-The backend doesn't support [websockets](../websockets.md).
+The backend doesn't support [websockets](../other/websockets.md).

docs/backends/javascript/fetch.md

@@ -1,4 +1,4 @@
-# JavaScript (Fetch) backend
+# Scala.js (Fetch) backend
 
 A JavaScript backend with web socket support. Implemented using the [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API).
 
@@ -61,7 +61,7 @@ your project:
 If you are on Cats Effect 2 (CE2) you will need to add the CE2 specific dependency instead:
 
 ```
-"com.softwaremill.sttp.client4" %%% "catsce2 % "@VERSION@"
+"com.softwaremill.sttp.client4" %%% "catsce2" % "@VERSION@"
 ```
 
 And create the backend instance:
@@ -151,13 +151,13 @@ val response: Task[Response[Observable[ByteBuffer]]] =
     .send(backend)
 ```      
 
-```{eval-rst}
-.. note:: Currently no browsers support passing a stream as the request body. As such, using the ``Fetch`` backend with a streaming request will result in it being converted into an in-memory array before being sent. Response bodies are returned as a "proper" stream.
+```{note}
+Currently no browsers support passing a stream as the request body. As such, using the `Fetch` backend with a streaming request will result in it being converted into an in-memory array before being sent. Response bodies are returned as a "proper" stream.
 ```
 
 ## Websockets
 
-The backend supports both regular and streaming [websockets](../../websockets.md).
+The backend supports both regular and streaming [websockets](../../other/websockets.md).
 
 ## Server-sent events