All tasks sometimes assigned to one thread on startup. #403
Description
I've run into an issue where on occasion after changing the NumStreamThreads config option we end up with all the tasks being assigned to one thread instead of spreading them equally.
Is this expected behavior? Is it a result of changing the number of stream threads rather than starting with more?
I've managed to recreate this issue intermittently, but not deterministically by running some changed example code with two topics. The output itself is nonsensical and only serves to prove it's processing them on different threads.
numbers-topic is set to ten partitions
sum-of-odd-numbers-topic is set to one partition
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Confluent.Kafka;
using Streamiz.Kafka.Net;
using Streamiz.Kafka.Net.SerDes;
using Streamiz.Kafka.Net.Stream;
using Streamiz.Kafka.Net.Table;
namespace sum
{
internal static class Program
{
static readonly String SUM_OF_ODD_NUMBERS_TOPIC = "sum-of-odd-numbers-topic";
static readonly String NUMBERS_TOPIC = "numbers-topic";
static string GetEnvironmentVariable(string var, string @default)
{
return Environment.GetEnvironmentVariable(var) ?? @default;
}
static async Task Main(string[] args)
{
CancellationTokenSource source = new CancellationTokenSource();
string boostrapserver = GetEnvironmentVariable("KAFKA_BOOTSTRAP_SERVER", "localhost:29092");
var config = new StreamConfig<StringSerDes, StringSerDes>();
config.NumStreamThreads = 5;
// Give the Streams application a unique name. The name must be unique in the Kafka cluster
// against which the application is run.
config.ApplicationId = "sum-example";
config.ClientId = "sum-example-client";
// Where to find Kafka broker(s).
config.BootstrapServers = boostrapserver;
config.AutoOffsetReset = Confluent.Kafka.AutoOffsetReset.Earliest;
config.CommitIntervalMs = 10 * 1000;
Topology t = GetTopology();
KafkaStream stream = new(t, config);
Console.CancelKeyPress += (o, e) => {
source.Cancel();
};
var prodConfig = new ProducerConfig
{
BootstrapServers = "localhost:29092",
};
using (var producer = new ProducerBuilder<string, string>(prodConfig).Build())
{
for (var i = 0; i < 100; i++)
{
await producer.ProduceAsync(NUMBERS_TOPIC, new Message<string, string>() { Key = i.ToString(), Value = i.ToString() });
}
}
await stream.StartAsync(source.Token);
}
static Topology GetTopology()
{
StreamBuilder builder = new();
// We assume the input topic contains records where the values are Integers.
// We don't really care about the keys of the input records; for simplicity, we assume them
// to be Integers, too, because we will re-key the stream later on, and the new key will be
// of type Integer.
var input = builder.Stream<string, string, StringSerDes, StringSerDes>(NUMBERS_TOPIC);
var sumOfOddNumbers = input
.MapValues((v, c) => v)
.SelectKey((k, v, c) => k)
.GroupByKey()
.Reduce((v1, v2) => v1 + v2);
//.ToTable();
sumOfOddNumbers
.ToStream()
.Map((k, v, c) => CreateKeyValuePair(k.ToString(), v.ToString()))
.To<StringSerDes, StringSerDes>(SUM_OF_ODD_NUMBERS_TOPIC, Partition);
return builder.Build();
}
private static KeyValuePair<string, string> CreateKeyValuePair(string key, string value)
{
Console.WriteLine($"Started processing {value.ToString()}");
Thread.Sleep(5000);
Console.WriteLine($"Finished processing {value.ToString()}");
return KeyValuePair.Create(key.ToString(), value.ToString());
}
private static Partition Partition(string topic, string key, string value, Partition sourcePartition, int numPartitions)
{
// if (sourcePartition.Value < numPartitions)
// return sourcePartition;
return Confluent.Kafka.Partition.Any;
}
}
}
Usually, the tasks will be assigned across the threads equally, like so:
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
Restoration took 1201ms for all tasks 0-1,0-0,1-0,1-1
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
Restoration took 1201ms for all tasks 0-5,0-4,1-4,1-5
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
Restoration took 1201ms for all tasks 0-3,0-2,1-2,1-3
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|6] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|7] Task 1-7 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|7] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
Restoration took 1201ms for all tasks 0-7,0-6,1-6,1-7
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|8] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|9] Task 1-9 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|9] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
Restoration took 1201ms for all tasks 0-9,0-8,1-8,1-9
Sometimes, it starts up like so, note 5 stream threads but all tasks on one of them, leading to the linear start->finish of the processing instead of it being interleaved:
info: Streamiz.Kafka.Net.KafkaStream[0]
stream-application[sum-example] Start creation of the stream application with this configuration:
Stream property:
client.id: sum-example-client
info: Streamiz.Kafka.Net.KafkaStream[0]
stream-application[sum-example] Start creation of the stream application with this configuration:
Stream property:
client.id: sum-example-client
num.stream.threads: 5
default.key.serdes: Streamiz.Kafka.Net.SerDes.StringSerDes
default.value.serdes: Streamiz.Kafka.Net.SerDes.StringSerDes
default.timestamp.extractor: Streamiz.Kafka.Net.Processors.Internal.FailOnInvalidTimestamp
commit.interval.ms: 10000
processing.guarantee: AT_LEAST_ONCE
transaction.timeout: 00:00:10
poll.ms: 100
max.poll.records: 500
max.poll.restoring.records: 1000
max.task.idle.ms: 0
buffered.records.per.partition: 2147483647
inner.exception.handler: System.Func`2[System.Exception,Streamiz.Kafka.Net.ExceptionHandlerResponse]
production.exception.handler: System.Func`2[Confluent.Kafka.DeliveryReport`2[System.Byte[],System.Byte[]],Streamiz.Kafka.Net.ProductionExceptionHandlerResponse]
deserialization.exception.handler: System.Func`4[Streamiz.Kafka.Net.ProcessorContext,Confluent.Kafka.ConsumeResult`2[System.Byte[],System.Byte[]],System.Exception,Streamiz.Kafka.Net.ExceptionHandlerResponse]
rocksdb.config.setter: System.Action`2[System.String,Streamiz.Kafka.Net.State.RocksDbOptions]
follow.metadata: False
state.dir: C:\Users\Mee\AppData\Local\Temp\streamiz-kafka-net
replication.factor: -1
windowstore.changelog.additional.retention.ms: 86400000
offset.checkpoint.manager:
metrics.interval.ms: 30000
metrics.recording.level: INFO
log.processing.summary: 00:01:00
metrics.reporter: System.Action`1[System.Collections.Generic.IEnumerable`1[Streamiz.Kafka.Net.Metrics.Sensor]]
expose.librdkafka.stats: False
start.task.delay.ms: 5000
parallel.processing: False
max.degree.of.parallelism: 8
statestore.cache.max.bytes: 5242880
application.id: sum-example
Client property:
bootstrap.servers: localhost:29092
Consumer property:
max.poll.interval.ms: 300000
enable.auto.commit: False
enable.auto.offset.store: False
allow.auto.create.topics: False
partition.assignment.strategy: range
auto.offset.reset: earliest
Producer property:
allow.auto.create.topics: False
partitioner: murmur2_random
Admin client property:
allow.auto.create.topics: False
info: Streamiz.Kafka.Net.Processors.Internal.StateDirectory[0]
Reading UUID from process file: 05e3e2a8-fae6-4cdd-b66e-5a9135f37983
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-0] Creating consumer client
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-1] Creating consumer client
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-2] Creating consumer client
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-3] Creating consumer client
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-4] Creating consumer client
info: Streamiz.Kafka.Net.KafkaStream[0]
stream-application[sum-example] State transition from CREATED to REBALANCING
info: Streamiz.Kafka.Net.KafkaStream[0]
stream-application[sum-example] Starting Streams client with this topology : Topologies:
Sub-topology: 1
Source: KSTREAM-SOURCE-0000000007 (topics: [KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition])
--> KSTREAM-AGGREGATE-0000000003
Processor: KSTREAM-AGGREGATE-0000000003 (stores: [KSTREAM-AGGREGATE-STATE-STORE-0000000004])
--> KTABLE-TOSTREAM-0000000008
<-- KSTREAM-SOURCE-0000000007
Processor: KTABLE-TOSTREAM-0000000008 (stores: [])
--> KSTREAM-MAP-0000000009
<-- KSTREAM-AGGREGATE-0000000003
Processor: KSTREAM-MAP-0000000009 (stores: [])
--> KSTREAM-SINK-0000000010
<-- KTABLE-TOSTREAM-0000000008
Sink: KSTREAM-SINK-0000000010 (topic: sum-of-odd-numbers-topic)
<-- KSTREAM-MAP-0000000009
Sub-topology: 0
Source: KSTREAM-SOURCE-0000000000 (topics: [numbers-topic])
--> KSTREAM-MAPVALUES-0000000001
Processor: KSTREAM-MAPVALUES-0000000001 (stores: [])
--> KSTREAM-KEY-SELECT-0000000002
<-- KSTREAM-SOURCE-0000000000
Processor: KSTREAM-KEY-SELECT-0000000002 (stores: [])
--> KSTREAM-FILTER-0000000006
<-- KSTREAM-MAPVALUES-0000000001
Processor: KSTREAM-FILTER-0000000006 (stores: [])
--> KSTREAM-SINK-0000000005
<-- KSTREAM-KEY-SELECT-0000000002
Sink: KSTREAM-SINK-0000000005 (topic: KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition)
<-- KSTREAM-FILTER-0000000006
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-0] Starting
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-0] State transition from CREATED to STARTING
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-1] Starting
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-1] State transition from CREATED to STARTING
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-2] Starting
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-2] State transition from CREATED to STARTING
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-3] Starting
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-3] State transition from CREATED to STARTING
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-4] Starting
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-4] State transition from CREATED to STARTING
info: Streamiz.Kafka.Net.Kafka.Internal.StreamsRebalanceListener[0]
New partitions assign requested : sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[0]],sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[1]],sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[2]],sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[3]],sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[4]],sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[5]],sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[6]],sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[7]],sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[8]],sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-repartition [[9]],numbers-topic [[0]],numbers-topic [[1]],numbers-topic [[2]],numbers-topic [[3]],numbers-topic [[4]],numbers-topic [[5]],numbers-topic [[6]],numbers-topic [[7]],numbers-topic [[8]],numbers-topic [[9]]
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-0] State transition from STARTING to PARTITIONS_ASSIGNED
info: Streamiz.Kafka.Net.Kafka.Internal.StreamsRebalanceListener[0]
Partition assignment took 00:00:01.0736748 ms.
Currently assigned active tasks: 0-9,0-8,0-7,0-6,0-5,0-4,0-3,0-2,0-1,0-0,1-0,1-2,1-4,1-6,1-8,1-9,1-1,1-3,1-5,1-7
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
stream-thread[sum-example-client-stream-thread-0] State transition from PARTITIONS_ASSIGNED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|9] Task 0-9 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|8] Task 0-8 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|7] Task 0-7 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|6] Task 0-6 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|5] Task 0-5 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|4] Task 0-4 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|3] Task 0-3 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|2] Task 0-2 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|1] Task 0-1 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[0|0] Task 0-0 state transition from CREATED to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|0] Task 1-0 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|0] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|2] Task 1-2 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|2] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|4] Task 1-4 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|4] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|6] Task 1-6 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|6] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|8] Task 1-8 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|8] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|9] Task 1-9 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|9] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|1] Task 1-1 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|1] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|3] Task 1-3 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|3] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|5] Task 1-5 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|5] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|7] Task 1-7 state transition from CREATED to RESTORING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|7] Restoration will start soon.
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[0]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[1]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[2]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[3]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[4]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[5]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[6]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[7]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[8]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.Internal.StoreChangelogReader[0]
Finished restoring changelog KSTREAM-AGGREGATE-STATE-STORE-0000000004 to store sum-example-KSTREAM-AGGREGATE-STATE-STORE-0000000004-changelog [[9]] with a total number of 0 records
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|0] Task 1-0 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|0] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|2] Task 1-2 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|2] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|4] Task 1-4 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|4] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|6] Task 1-6 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|6] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|8] Task 1-8 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|8] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|9] Task 1-9 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|9] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|1] Task 1-1 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|1] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|3] Task 1-3 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|3] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|5] Task 1-5 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|5] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|7] Task 1-7 state transition from RESTORING to RUNNING
info: Streamiz.Kafka.Net.Processors.StreamTask[0]
stream-task[1|7] Restored and ready to run
info: Streamiz.Kafka.Net.Processors.StreamThread[0]
Restoration took 1359ms for all tasks 0-9,0-8,0-7,0-6,0-5,0-4,0-3,0-2,0-1,0-0,1-0,1-2,1-4,1-6,1-8,1-9,1-1,1-3,1-5,1-7
Started processing 5555555555555
Finished processing 5555555555555
Started processing 1818181818181818181818181818
Finished processing 1818181818181818181818181818