| copyright |
|
||
|---|---|---|---|
| lastupdated | 2018-08-08 |
{:new_window: target="_blank"} {:shortdesc: .shortdesc} {:screen: .screen} {:codeblock: .codeblock} {:pre: .pre}
{: #faqs}
Answers to common questions about the {{site.data.keyword.IBM}} {{site.data.keyword.messagehub}} service. {: shortdesc}
{: #topic_admin}
If you're using a Kafka client at 0.11 or later, or Kafka Streams at 0.10.2.0 or later, you can use APIs to create and delete topics. We've put some restrictions on the settings allowed when you create topics. Currently, you can modify the following settings only:
- cleanup.policy
- Set to
delete(default),compactordelete,compact**Note:** If the cleanup policy is
compactonly, we automatically adddelete, but disable deletion based on time. Messages in the topic are compacted up to 1 GB before being deleted. - retention.ms
- The default retention period is 24 hours. The minimum is 1 hour and the maximum is
30 days. Specify this value as multiples of hours.
**Note:** In the Enterprise plan, you can set this to any value.
- retention.bytes
- The maximum size a partition (which consists of log segments) can grow to before we discard old log segments to free up space.
**Note:** Enterprise plan only. Set to any value larger than 1 MB.
- segment.bytes
- The segment file size for the log.
**Note:** Enterprise plan only. Set to any value larger than 100 kB.
- segment.index.bytes
- The size of the index that maps offsets to file positions.
**Note:** Enterprise plan only. Set to any value between 100 kB and 2 GB.
- segment.ms
- The period of time after which Kafka will force the log to roll even if the segment file isn't full.
**Note:** Enterprise plan only. Set to any value between 5 minutes and 30 days
How long does {{site.data.keyword.messagehub}} set the log retention window for the consumer offsets topic?
{: #offsets } {{site.data.keyword.messagehub}} retains consumer offsets for 7 days. This corresponds to the Kafka configuration offsets.retention.minutes.
Offset retention is system-wide so you cannot set it at an individual topic level. All consumer groups get only 7 days of stored offsets even if using a topic with a log retention that has been increased to the maximum of 30 days.
{: #availability}
If you write {{site.data.keyword.messagehub}} apps, use this information to understand what normal {{site.data.keyword.messagehub}} availability behavior is and what your apps are expected to handle.
{: #api_availability }
As part of the regular operation of {{site.data.keyword.messagehub}}, the nodes of the Kafka clusters are occasionally restarted. In some cases, your apps will be aware as the cluster reassigns resources. Write your apps to be resilient to these changes and to be able to reconnect and retry operations.
{: #bridge_availability }
Write your apps to handle the possibility that bridges might restart from time to time.
{: #max_message_size }
{{site.data.keyword.messagehub}}'s maximum message size is 1 MB, which is the Kafka default.
{: #replication }
{{site.data.keyword.messagehub}} is configured to provide strong availability and durability. The following configuration settings apply to all topics and cannot be changed:
- replication.factor = 3
- min.insync.replicas = 2
{: #billing }
{{site.data.keyword.messagehub}} on the Standard plan regularly samples a user's topic count and {{site.data.keyword.Bluemix_notm}} records the maximum sample value each day. {{site.data.keyword.messagehub}} bills for the maximum number of concurrent partitions seen and for the sum of messages that are sent and received daily.
For example, if you create and delete 1 topic 10 times in a day, you are charged for a maximum of 1 topic. However, if you create 10 topics and delete them, you might be charged for either 0 or 10 topics depending when the sampling takes place.
{{site.data.keyword.messagehub}} bills either for each message or for each 64 k. A message up to 64 k counts as 1 billable message. Messages larger than 64 k count as the following number of billable messages: message_size ÷ 64 k.
{: #REST_restart }
The Kafka REST API restarts once a day for a short period of time.
During this period, the Kafka REST API might become unavailable. If this happens, you are recommended to retry your request. After the REST API has restarted, you will have to recreate your Kafka consumer instances. If this is the case, the REST API returns the following JSON:
{:screen}
What are the differences between the {{site.data.keyword.messagehub}} Standard and {{site.data.keyword.messagehub}} Enterprise plans?
{: #plan_compare }
To find out more information about the two different {{site.data.keyword.messagehub}} plans, see Choosing your plan.
{: #disaster_recovery }
Currently, it is the responsibility of the user to manage their own {{site.data.keyword.messagehub}} disaster recovery. {{site.data.keyword.messagehub}} data can be replicated between an {{site.data.keyword.messagehub}} instance in one region and another instance in a different region. However, the user is responsible for provisioning a remote {{site.data.keyword.messagehub}} instance and managing the replication.
The user is also responsible for the backup of message payload data. Although this data is replicated across multiple Kafka brokers within a cluster, which protects against the majority of failures, this replication does not cover a region-wide failure.
Topic names are backed up by {{site.data.keyword.messagehub}}.