Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-8249

Kinesis Producer didnt configure region

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.0
    • 1.4.1, 1.5.0
    • Connectors / Kinesis
    • None

    Description

      Hi,

      setting this configurations to FlinkKinesisProducer:

      properties.put(AWSConfigConstants.AWS_REGION, "eu-west-1");
      properties.put(AWSConfigConstants.AWS_ACCESS_KEY_ID, "accessKey");
      properties.put(AWSConfigConstants.AWS_SECRET_ACCESS_KEY, "secretKey");
      

      is throwing this error:

      17/12/13 10:50:11 ERROR LogInputStreamReader: [2017-12-13 10:50:11.290786] [0x000057ba][0x00007f31cbce5780] [error] [main.cc:266] Could not configure the region. It was not given in the config and we were unable to retrieve it from EC2 metadata.
      17/12/13 10:50:12 ERROR KinesisProducer: Error in child process
      org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.IrrecoverableError: Child process exited with code 1
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:525)
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:497)
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:475)
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63)
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700441] [0x000057c4][0x00007ffb152b5780] [error] [AWS Log: ERROR](CurlHttpClient)Curl returned error code 28
      17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700521] [0x000057c4][0x00007ffb152b5780] [error] [AWS Log: ERROR](EC2MetadataClient)Http request to Ec2MetadataService failed.
      

      making some investigations the region is never setted and i think this is the reason:
      in this commit: https://github.com/apache/flink/commit/9ed5d9a180dcd871e33bf8982434e3afd90ed295#diff-f3c6c35f3b045df8408b310f8f8a6bc7

      -		KinesisProducerConfiguration producerConfig = new KinesisProducerConfiguration();
      -		producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
      +		// check and pass the configuration properties
      +		KinesisProducerConfiguration producerConfig = KinesisConfigUtil.validateProducerConfiguration(configProps);
       		producerConfig.setCredentialsProvider(AWSUtil.getCredentialsProvider(configProps));
      

      this line was removed

      producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
      

      cc tzulitai, phoenixjiangnan

      Attachments

        Activity

          People

            Unassigned Unassigned
            eskabetxe João Boto
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: