Skip to main content

Deserialization error causing Kafka Streams to stop working even in presence of DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG property

If you are getting an exception during your kafka stream thread processing for which stack trace resembles what is below and you are wondering why your DeserializationExceptionHandler is not being invoked (which is written to ignore a deserialization error on a particular record), you are probably like me who have spent hours trying to figure out that.

Ultimately I had to look closely on the stack trace and underlying Kafka streams library, I figured out this error is not occurring when a record is being processed from Kafka topic. Instead, it is being processed when a record from a State store is being retrieved (Notice the StateSerdes in the exception stack trace).

Kafka doesn't use the handler in DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG when it reads the RocksDB files (see that the stacktrace mentions the class StateSerdes). That's why it works fine for records coming from the source topic, but fails when deserializing the data in the table.

You would need to look at why the data gets corrupted in the State store itself to reproduce the error.

Failed to process stream task 0_4 due to the following error:

java.lang.RuntimeException: Error deserializing data '' for object 

at org.apache.kafka.streams.state.internals.ValueAndTimestampDeserializer.deserialize(ValueAndTimestampDeserializer.java:54) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.state.internals.ValueAndTimestampDeserializer.deserialize(ValueAndTimestampDeserializer.java:27) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.state.StateSerdes.valueFrom(StateSerdes.java:160) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.outerValue(MeteredKeyValueStore.java:267) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.get(MeteredKeyValueStore.java:148) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.ProcessorContextImpl$KeyValueStoreReadWriteDecorator.get(ProcessorContextImpl.java:464) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.kstream.internals.KTableSource$KTableSourceProcessor.process(KTableSource.java:102) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:117) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:201) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:180) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:133) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:87) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:363) ~[kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.AssignedStreamsTasks.process(AssignedStreamsTasks.java:199) [kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.TaskManager.process(TaskManager.java:425) [kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:912) [kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:819) [kafka-streams-2.3.1.jar!/:?]

at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:788) [kafka-streams-2.3.1.jar!/:?]

Caused by: java.io.EOFException

at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:126) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.io.ExtendedJsonDecoder.readString(ExtendedJsonDecoder.java:249) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:220) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:372) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:145) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:126) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:212) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:202) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:133) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:126) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:212) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:202) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:133) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:126) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:103) ~[avro-1.9.0.11p.jar!/:1.9.0.11p]

Labels:

Comments

Post a Comment

Popular posts from this blog

Customers Service Executive Job in Abu Dhabi United Arab Emirates

Customers Service Executive (Abu Dhabi) - Parking and Removing / Relocation experience mandatory. Checking & verifying with sales dept for freight rates for processing import shipment • Coordinating with the customs clearance assistant and airlines for the collection of Delivery Orders from DNATA. • Handling sea freight import FCL & LCL containers • Liaising with shipping lines/agents, DPA, Customs authority for LCL/FCL or co-load type of shipments • Monitor various shipping movement and keep a track of all shipments & vessel arrival. • To make import bookings for warehouse & set priorities with regular follow-up for urgent deliveries • Ensure timely availability of shipments to customers within the time frame. • Co-ordination with overseas partner & airline agents. • Follow-up for NOC (No Objection Letter) from the liner agents and collect MR number from DPA • Sending arrival notification to customers • Checking & verifying with sales dept for freight rates f
Read more

Key Account Manager (Dubai) Job at Air Arabia Dubai United Arab Emirates

Job Description Air Arabia is not only committed to providing affordable air travel but is also dedicated to uplifting the lives of those who are less fortunate. Taking responsibility and lead on social needs of local and international communities has been part of our success. Job Purpose To achieve the commercial objectives and targets in the assigned area/territory from different channels: building effective business relationships, enhancing brand visibility and promoting Company’s products and services. Carries marketing campaigns and promotions, and increases the market share in line with set measures and Company’s adopted policies and procedures. Key Result Responsibilities Achieves and exceeds the set sales targets in the assigned area/territory; constantly reviews financial data to evaluate performance; identifies shortages, determines root causes, recommends changes to respective Manager and implements corrective measures accordingly. Identifies new opportunities to manage/maxi
Post a Comment
Read more

Assistant Sales Manager Tourism Dusk Tourism LLC Dubai, United Arab Emirates

The Sales Manager’s responsibilities include developing strategies to improve sales, effectively handling customer complaints. You should also be able to delegate tasks to the sales staff in the absence . To be successful as an Sales Manager, you should demonstrate exceptional leadership skills and ensure that sales staff comply with company policies and guidelines. Ultimately, a top-performing Sales Manager should be able to achieve excellent customer service at all times. Sales Manager Responsibilities: Continually meeting or exceeding sales quotas. Supervising and guiding the sales team as well as providing incentives to motivate staff to achieve sales targets. Monitoring the performance of the sales team. Building and maintaining good working relationships with customers. Identifying opportunities and strategies to increase sales. Regularly attending sales meetings and training sessions. Sales Manager Requirements: Bachelor’s degree in Marketing, Communications, or related field is
Post a Comment
Read more