Apache Flink: Kafka Connector In Python Streaming Api, "cannot Load User Class"

I am trying out Flink's new Python streaming API and attempting to run my script with ./flink-1.6.1/bin/pyflink-stream.sh examples/read_from_kafka.py. The python script is fairly s

Solution 1:

You are using wrong Kafka consumer here. In your code, it is FlinkKafkaConsumer09, but the lib you are using is flink-connector-kafka-0.11_2.11-1.6.1.jar, which is for FlinkKafkaConsumer011. Try to replace FlinkKafkaConsumer09 with this FlinkKafkaConsumer011, or use the lib file flink-connector-kafka-0.9_2.11-1.6.1.jar instead of current one.

Solution 2:

I guest the jar file may has built-in import or dependencies,so the three jar files isn't enough.As to how to find out java jar dependent relations,that is what java maven do. You can see the official website "project build setup" for help. In my case,i follow official java project setup,use "from org.apache.flink.streaming.connectors.kafka import FlinkKafkaConsumer" and add dependency " org.apache.flink flink-clients_2.11 1.8.0 " to pom.xml,then i can output kafka records to stdout now with the Python API.