[STORM-2264] OpaqueTridentKafkaSpout failing after STORM-2216 - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Critical
Resolution: Fixed
Affects Version/s: 2.0.0, 1.1.0, 1.0.3
Fix Version/s: 2.0.0, 1.1.0, 1.0.3
Component/s: storm-kafka
Labels:
None

Epic Link:
Release Apache Storm 1.1.0

Description

I've seen OpaqueTridentKafkaSpout failing after ~~STORM-2216~~.

java.lang.RuntimeException: Unexpected character (G) at position 1.
	at org.apache.storm.trident.topology.state.TransactionalState.getData(TransactionalState.java:172) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.trident.topology.state.RotatingTransactionalState.sync(RotatingTransactionalState.java:165) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.trident.topology.state.RotatingTransactionalState.<init>(RotatingTransactionalState.java:46) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.trident.spout.TridentSpoutCoordinator.prepare(TridentSpoutCoordinator.java:57) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.topology.BasicBoltExecutor.prepare(BasicBoltExecutor.java:43) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:84) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:93) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:45) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.Utils$6.run(Utils.java:2179) [storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
Caused by: org.apache.storm.shade.org.json.simple.parser.ParseException
	at org.apache.storm.shade.org.json.simple.parser.Yylex.yylex(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.shade.org.json.simple.parser.JSONParser.nextToken(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.shade.org.json.simple.parser.JSONParser.parse(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.shade.org.json.simple.parser.JSONParser.parse(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.shade.org.json.simple.parser.JSONParser.parse(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.shade.org.json.simple.JSONValue.parseWithException(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.trident.topology.state.TransactionalState.getData(TransactionalState.java:165) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	... 9 more

There's no bug on ~~STORM-2216~~, but there has been limitation on TransactionalState which doesn't have type information and uses simple-json to serde, in result it can't serde GlobalPartitionInformation properly.

Previously we just ignored parse error but ~~STORM-2216~~ exposes the issue.

We could make TransactionalState ignoring parse error again, but anyway it has been not intended behavior so better to be fixed properly.

Attachments

Issue Links

is duplicated by

STORM-2265 Incorrectly Serialized JSON in TransactionalState causes Worker to Die

Closed

links to

GitHub Pull Request #1853

Activity

People

Assignee:: Jungtaek Lim

Reporter:: Jungtaek Lim

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 27/Dec/16 23:08

Updated:: 09/Jan/17 00:48

Resolved:: 09/Jan/17 00:48

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

1h 10m