Skip to content

Commit 68d32b9

Browse files
jenkinsjenkins
authored andcommitted
Merge commit 'db5c7afacc26abae73fe10625eeeeb8e192760d6'
2 parents 9a0f2ea + db5c7af commit 68d32b9

60 files changed

Lines changed: 1975 additions & 620 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.rst

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,42 @@ Note that ``RB_ID=#`` and ``PHAB_ID=#`` correspond to associated message in comm
77
Unreleased
88
----------
99

10+
19.6.0
11+
------
12+
13+
Added
14+
~~~~~
15+
16+
* inject-modules: Introduce a `StackClientModuleTrait` to aid in configuring modules for generic
17+
Finagle Stack Clients. ``PHAB_ID=D324891``
18+
19+
* finatra-http: Add `c.t.inject.server.InMemoryStatsReceiverUtility` which allows for testing
20+
assertions on metrics captured within an embedded server's `InMemoryStatsReceiver`. Update the
21+
Kafka tests and utilities to use the `InMemoryStatsReceiverUtility` and mark the
22+
`c.t.finatra.kafka.test.utilsInMemoryStatsUtil` as deprecated. ``PHAB_ID=D316806``
23+
24+
Changed
25+
~~~~~~~
26+
27+
* finatra-http: Removed deprecated `response_size` stat from `c.t.finatra.http.filters.StatsFilter`.
28+
``PHAB_ID=D328254``
29+
30+
* finatra-kafka: Update finatra exported metrics to contains KafkaMetrics 'rocksdb-window-state-id'.
31+
``PHAB_ID=D326320``
32+
33+
* finatra-kafka: Deprecate in `c.t.finatra.kafka.consumers.FinagleKafkaConsumer`.
34+
Add `c.t.finatra.kafka.consumers.FinagleKafkaConsumer.buildClient` and
35+
`c.t.finatra.kafka.producters.FinagleKafkaProducer.buildClient`. ``PHAB_ID=D321699``
36+
37+
Fixed
38+
~~~~~
39+
40+
* finatra: Add an explicit dependency on `com.sun.activation` to allow for using
41+
Finatra with JDK 11. This fixes #484. ``PHAB_ID=D328724``
42+
1043
19.5.1
1144
------
45+
1246
Fixed
1347
~~~~~
1448

build.sbt

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ import scoverage.ScoverageKeys
44
concurrentRestrictions in Global += Tags.limit(Tags.Test, 1)
55

66
// All Twitter library releases are date versioned as YY.MM.patch
7-
val releaseVersion = "19.5.1"
7+
val releaseVersion = "19.6.0"
88

99
lazy val buildSettings = Seq(
1010
version := releaseVersion,
@@ -412,6 +412,8 @@ lazy val injectServerTestJarSources =
412412
"com/twitter/inject/server/EmbeddedTwitterServer",
413413
"com/twitter/inject/server/FeatureTest",
414414
"com/twitter/inject/server/FeatureTestMixin",
415+
"com/twitter/inject/server/InMemoryStats",
416+
"com/twitter/inject/server/InMemoryStatsReceiverUtility",
415417
"com/twitter/inject/server/package")
416418
lazy val injectServer = (project in file("inject/inject-server"))
417419
.settings(projectSettings)
@@ -636,6 +638,7 @@ lazy val http = project
636638
moduleName := "finatra-http",
637639
ScoverageKeys.coverageExcludedPackages := "<empty>;.*ScalaObjectHandler.*;.*NonValidatingHttpHeadersResponse.*;com\\.twitter\\.finatra\\..*package.*;.*ThriftExceptionMapper.*;.*HttpResponseExceptionMapper.*;.*HttpResponseException.*",
638640
libraryDependencies ++= Seq(
641+
"com.sun.activation" % "javax.activation" % "1.2.0",
639642
"com.github.spullara.mustache.java" % "compiler" % versions.mustache exclude("com.google.guava", "guava"),
640643
"com.twitter" %% "finagle-exp" % versions.twLibVersion,
641644
"com.twitter" %% "finagle-http" % versions.twLibVersion,

doc/src/sphinx/user-guide/app/index.rst

Lines changed: 107 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,15 @@
33
Creating an injectable |c.t.app.App|_
44
=====================================
55

6+
Lifecycle
7+
---------
8+
9+
If you haven't already, take a look at the |c.t.app.App|_
10+
`lifecycle documentation <../getting-started/lifecycle.html#c-t-app-app-lifecycle>`__.
11+
12+
Getting Started
13+
---------------
14+
615
To create an injectable |c.t.app.App|_, first depend on the `inject-app` library. Then use the
716
`inject framework <../getting-started/framework.html#inject>`__ to create an injectable App. Finatra
817
provides an injectable version of the |c.t.app.App|_ trait: |c.t.inject.app.App|_.
@@ -16,24 +25,49 @@ in a |c.t.app.App|_ with support for `modules <../getting-started/modules.html>`
1625
Scala Example
1726
-------------
1827

28+
Given a `module <../getting-started/modules.html>`__ definition:
29+
30+
.. code:: scala
31+
32+
import com.twitter.inject.TwitterModule
33+
34+
object MyModule1 extends TwitterModule {
35+
36+
@Singleton
37+
@Provides
38+
def providesFooService: FooService = ???
39+
}
40+
41+
You could define an |c.t.inject.app.App|_:
42+
1943
.. code:: scala
2044
2145
import com.google.inject.Module
22-
import com.twitter.inject.app.App
2346
import com.twitter.inject.Logging
47+
import com.twitter.inject.app.App
48+
import com.twitter.inject.modules.LoggingModule
2449
2550
object MyAppMain extends MyApp
2651
2752
class MyApp extends App with Logging {
2853
2954
override val modules: Seq[Module] = Seq(
55+
LoggingModule,
3056
MyModule1)
3157
3258
override protected def run(): Unit = {
3359
// Core app logic goes here.
60+
val fooService = injector.instance[FooService]
61+
???
3462
}
3563
}
3664
65+
.. tip::
66+
67+
We include the `c.t.inject.modules.LoggingModule <https://github.com/twitter/finatra/blob/develop/inject/inject-modules/src/main/scala/com/twitter/inject/modules/LoggerModule.scala>`__ to attempt installation of the
68+
`SLF4JBridgeHandler <https://www.slf4j.org/api/org/slf4j/bridge/SLF4JBridgeHandler.html>`__ here as an example of
69+
how to `bridge legacy APIs <https://www.slf4j.org/legacy.html>`__.
70+
3771
Then to test:
3872

3973
.. code:: scala
@@ -61,21 +95,38 @@ Java Example
6195
import com.google.inject.Module;
6296
6397
import com.twitter.inject.app.AbstractApp;
98+
import com.twitter.inject.modules.LoggerModule$;
6499
65100
public class MyApp extends AbstractApp {
66101
67102
@Override
68103
public Collection<Module> javaModules() {
69104
return Collections.<Module>singletonList(
105+
LoggerModule$.MODULE$,
70106
MyModule1$.MODULE$);
71107
}
72108
73109
@Override
74110
public void run() {
75111
// Core app logic goes here.
112+
FooService fooService =
113+
injector().instance(FooService.class);
76114
}
77115
}
78116
117+
Then create a "main" class:
118+
119+
.. code:: java
120+
121+
final class MyAppMain {
122+
private MyAppMain() {
123+
}
124+
125+
public static void main(String[] args) {
126+
new MyApp().main(args);
127+
}
128+
}
129+
79130
See the Finatra `examples <https://github.com/twitter/finatra/tree/develop/examples>`__ for detailed examples with tests.
80131

81132
`App#run`
@@ -87,6 +138,61 @@ The Finatra `Startup Lifecycle <../getting-started/lifecycle.html#startup>`__ en
87138
will be properly configured before access and provides the ``#run`` method as the function for implementing
88139
the app.
89140

141+
Using Injection
142+
---------------
143+
144+
The |c.t.inject.app.App|_ exposes access to the configured `c.t.inject.Injector <https://github.com/twitter/finatra/blob/develop/inject/inject-core/src/main/scala/com/twitter/inject/Injector.scala>`__ which can be used to obtain instances from
145+
the object graph.
146+
147+
.. note::
148+
149+
You should take care to only access the injector in the `App#run` function as this is the correct place in the
150+
`lifecycle <../getting-started/lifecycle.html#c-t-app-app-lifecycle>`__ where you are ensured to have a fully
151+
configured `c.t.inject.Injector`. Accessing the `c.t.inject.Injector` too early in the lifecycle (before it is
152+
configured) will result in an Exception being thrown.
153+
154+
An Example of Handling Signals
155+
------------------------------
156+
157+
There may be cases where you want your application to handle an
158+
`IPC signal <https://en.wikipedia.org/wiki/Signal_(IPC)>`__ instead of closing normally
159+
once the code execution is done, e.g., handling an `INT` (Ctrl-C) or `TSTP` (Ctrl-Z) signal.
160+
161+
You can use the
162+
`com.twitter.util.HandleSignal <https://github.com/twitter/util/blob/aaa3d6e2bf37a3bd565a8c51187fd9b6db8a0b25/util-core/src/main/scala/com/twitter/util/Signal.scala#L75>`__ utility to apply a callback to run on receiving
163+
the signal.
164+
165+
.. note::
166+
167+
Please consult the scaladocs for `com.twitter.util.HandleSignal <https://github.com/twitter/util/blob/aaa3d6e2bf37a3bd565a8c51187fd9b6db8a0b25/util-core/src/main/scala/com/twitter/util/Signal.scala#L75>`__
168+
to make sure you are aware of the limitations of the code in handling signals.
169+
170+
For example, to exit the application upon receiving an `INT` signal:
171+
172+
.. code:: scala
173+
174+
import com.google.inject.Module
175+
import com.twitter.inject.app.App
176+
import com.twitter.inject.Logging
177+
178+
object MyAppMain extends MyApp
179+
180+
class MyApp extends App with Logging {
181+
182+
override val modules: Seq[Module] = Seq(
183+
MyModule1)
184+
185+
HandleSignal("INT") { signal =>
186+
exitOnError(s"Process is being terminated with signal $signal.")
187+
}
188+
189+
override protected def run(): Unit = {
190+
// Core app logic goes here.
191+
val fooService = injector.instance[FooService]
192+
???
193+
}
194+
}
195+
90196
.. |c.t.app.App| replace:: `util-app App`
91197
.. _c.t.app.App: https://github.com/twitter/util/blob/develop/util-app/src/main/scala/com/twitter/app/App.scala
92198

doc/src/sphinx/user-guide/getting-started/framework.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ Modules in `inject` include:
4343
Creating an injectable App or TwitterServer
4444
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4545

46-
See: `Building an injectable c.t.app.App <../app/index.html>`__ or `Building an injectable c.t.server.TwitterServer <.../twitter_server/html>`__.
46+
See: `Building an injectable c.t.app.App <../app/index.html>`__ or `Building an injectable c.t.server.TwitterServer <../twitter-server/index.html>`__.
4747

4848
.. important::
4949

doc/src/sphinx/user-guide/json/index.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,4 +117,4 @@ The following `Jackson annotations <https://github.com/FasterXML/jackson-annotat
117117
Known limitations
118118
-----------------
119119

120-
- ``@JsonCreator`` is not supported. Finatra ignores this annotation (see: `#447 <https://github.com/twitter/finatra/issues/447>`).
120+
- ``@JsonCreator`` is not supported. Finatra ignores this annotation (see: `#447 <https://github.com/twitter/finatra/issues/447>`__).

doc/src/sphinx/user-guide/kafka-streams/examples.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
Examples
44
========
55

6-
The `integration tests <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/test/scala/com/twitter/unittests/integration>`__ serve as a good collection of example Finatra Kafka Streams servers.
6+
The `integration tests <https://github.com/twitter/finatra/tree/develop/kafka-streams/kafka-streams/src/test/scala/com/twitter/finatra/kafkastreams/integration>`__ serve as a good collection of example Finatra Kafka Streams servers.
77

88
Word Count Server
99
-----------------
@@ -52,4 +52,4 @@ We can then expose a Thrift endpoint enabling clients to directly query the stat
5252
}
5353
}
5454
55-
In this example, ``WordCountQueryService`` is an underlying Thrift service.
55+
In this example, ``WordCountQueryService`` is an underlying Thrift service.

doc/src/sphinx/user-guide/kafka-streams/index.rst

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Finatra has native integration with `Kafka Streams <https://kafka.apache.org/doc
88
Features
99
--------
1010

11-
- Intuitive `DSL <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/kafkastreams/internal/utils/FinatraDslV2Implicits.scala>`__ for topology creation, compatible with the `Kafka Streams DSL <https://kafka.apache.org/21/documentation/streams/developer-guide/dsl-api.html>`__
11+
- Intuitive `DSL <https://github.com/twitter/finatra/tree/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/kafkastreams/dsl>`__ for topology creation, compatible with the `Kafka Streams DSL <https://kafka.apache.org/21/documentation/streams/developer-guide/dsl-api.html>`__
1212
- Full Kafka Streams metric integration, exposed as `TwitterServer Metrics <https://twitter.github.io/twitter-server/Features.html#metrics>`__
1313
- `RocksDB integration <#rocksdb>`__
1414
- `Queryable State <#queryable-state>`__
@@ -23,15 +23,15 @@ a fully functional service can be written by simply configuring the Kafka Stream
2323
Transformers
2424
~~~~~~~~~~~~
2525

26-
Implement custom `transformers <https://kafka.apache.org/21/javadoc/org/apache/kafka/streams/kstream/Transformer.html>`__ using `FinatraTransformer <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/streams/transformer/FinatraTransformer.scala>`__.
26+
Implement custom `transformers <https://kafka.apache.org/21/javadoc/org/apache/kafka/streams/kstream/Transformer.html>`__ using `FinatraTransformer <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/kafkastreams/transformer/FinatraTransformer.scala>`__.
2727

2828
Aggregations
2929
^^^^^^^^^^^^
3030

3131
There are several included aggregating transformers, which may be used when configuring a ``StreamsBuilder``
32+
+ ``aggregate``
3233
+ ``sample``
33-
+ ``sum``
34-
+ ``compositeSum``
34+
+ ``sum``
3535

3636
Stores
3737
------
@@ -44,13 +44,13 @@ In addition to using `state stores <https://kafka.apache.org/21/javadoc/org/apac
4444
Queryable State
4545
~~~~~~~~~~~~~~~
4646

47-
Finatra Kafka Streams supports directly querying state from a store. This can be useful for creating a service that serves data aggregated within a local Topology. You can use `static partitioning <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams-static-partitioning/src/main/scala/com/twitter/finatra/streams/partitioning/StaticPartitioning.scala>`__ to query an instance deterministically known to hold a key.
47+
Finatra Kafka Streams supports directly querying state from a store. This can be useful for creating a service that serves data aggregated within a local Topology. You can use `static partitioning <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams-static-partitioning/src/main/scala/com/twitter/finatra/kafkastreams/partitioning/StaticPartitioning.scala>`__ to query an instance deterministically known to hold a key.
4848

4949
See how queryable state is used in the following `example <examples.html#queryable-state>`__.
5050

5151
Queryable Stores
5252
^^^^^^^^^^^^^^^^
5353

54-
- `QueryableFinatraKeyValueStore <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/streams/query/QueryableFinatraKeyValueStore.scala>`__
55-
- `QueryableFinatraWindowStore <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/streams/query/QueryableFinatraWindowStore.scala>`__
56-
- `QueryableFinatraCompositeWindowStore <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/streams/query/QueryableFinatraCompositeWindowStore.scala>`__
54+
- `QueryableFinatraKeyValueStore <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/kafkastreams/query/QueryableFinatraKeyValueStore.scala>`__
55+
- `QueryableFinatraWindowStore <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/kafkastreams/query/QueryableFinatraWindowStore.scala>`__
56+
- `QueryableFinatraCompositeWindowStore <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/kafkastreams/query/QueryableFinatraCompositeWindowStore.scala>`__

doc/src/sphinx/user-guide/kafka-streams/testing.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@
33
Testing
44
=======
55

6-
Finatra Kafka Streams includes tooling that simplifies the process of writing highly testable services. See `TopologyFeatureTest <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/test/scala/com/twitter/finatra/streams/tests/TopologyFeatureTest.scala>`__, which includes a `FinatraTopologyTester <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/test/scala/com/twitter/finatra/streams/tests/FinatraTopologyTester.scala>`__ that integrates Kafka Streams' `TopologyTestDriver <https://kafka.apache.org/21/javadoc/org/apache/kafka/streams/TopologyTestDriver.html>`__ with a `KafkaStreamsTwitterServer <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/kafkastreams/KafkaStreamsTwitterServer.scala>`__.
6+
Finatra Kafka Streams includes tooling that simplifies the process of writing highly testable services. See `TopologyFeatureTest <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/test/scala/com/twitter/finatra/kafkastreams/test/TopologyFeatureTest.scala>`__, which includes a `FinatraTopologyTester <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/test/scala/com/twitter/finatra/kafkastreams/test/FinatraTopologyTester.scala>`__ that integrates Kafka Streams' `TopologyTestDriver <https://kafka.apache.org/21/javadoc/org/apache/kafka/streams/TopologyTestDriver.html>`__ with a `KafkaStreamsTwitterServer <https://github.com/twitter/finatra/blob/develop/kafka-streams/kafka-streams/src/main/scala/com/twitter/finatra/kafkastreams/KafkaStreamsTwitterServer.scala>`__.

doc/src/sphinx/user-guide/logging/index.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ transitively provide SLF4J bridges for the following logging providers:
3030
Note, if you are not using `c.t.inject.server.TwitterServer` or a subclass, e.g., you are
3131
building a command line application directly with `c.t.inject.app.App`, you can include the
3232
`LoggingModule <https://github.com/twitter/finatra/blob/develop/inject/inject-modules/src/main/scala/com/twitter/inject/modules/LoggerModule.scala>`__
33-
to attempt installation the `SLF4JBridgeHandler <https://www.slf4j.org/api/org/slf4j/bridge/SLF4JBridgeHandler.html>`__.
33+
to attempt installation of the `SLF4JBridgeHandler <https://www.slf4j.org/api/org/slf4j/bridge/SLF4JBridgeHandler.html>`__.
3434

3535
For more information on the SLF4J bridges see the SLF4J
3636
`Bridging legacy APIs <https://www.slf4j.org/legacy.html>`__ documentation.

doc/src/sphinx/user-guide/twitter-server/index.rst

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,19 @@
33
Creating an injectable |TwitterServer|_
44
=======================================
55

6+
Lifecycle
7+
---------
8+
9+
If you haven't already, take a look at the |TwitterServer|_
10+
`lifecycle documentation <../getting-started/lifecycle.html#c-t-server-twitterserver-lifecycle>`__.
11+
You may also want to look at the |c.t.app.App|_
12+
`lifecycle documentation <../getting-started/lifecycle.html#c-t-app-app-lifecycle>`__ as well since
13+
|c.t.server.TwitterServer|_ is an extension of |c.t.app.App|_ and thus inherits the |c.t.app.App|_
14+
lifecycle.
15+
16+
Getting Started
17+
---------------
18+
619
To create an injectable |c.t.server.TwitterServer|_, first depend on the `inject-server` library. We
720
also recommend using `Logback <https://logback.qos.ch/>`__ as your
821
`SLF4J <https://www.slf4j.org/manual.html>`__ implementation. E.g.,
@@ -226,6 +239,9 @@ For more information on the server lifecycle see the `Application and Server Lif
226239
section which contains details around the order of lifecycle events during `startup <../getting-started/lifecycle.html#startup>`__
227240
and considerations during `shutdown <../getting-started/lifecycle.html#shutdown>`__.
228241

242+
.. |c.t.app.App| replace:: `util-app App`
243+
.. _c.t.app.App: https://github.com/twitter/util/blob/develop/util-app/src/main/scala/com/twitter/app/App.scala
244+
229245
.. |c.t.inject.server.TwitterServer| replace:: ``c.t.inject.server.TwitterServer``
230246
.. _c.t.inject.server.TwitterServer: https://github.com/twitter/finatra/blob/develop/inject/inject-server/src/main/scala/com/twitter/inject/server/TwitterServer.scala
231247

0 commit comments

Comments
 (0)