Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-930

samza-autoscaling YarnUtils json deserialisation fails with list values

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.10.1
    • 0.10.1
    • None
    • None

    Description

      Current:

      The getRunningAppId method in org.apache.samza.autoscaling.utils.YarnUtil attempts to deserialise the json from this endpoint /ws/v1/cluster/apps on the resource manager into the following shape: <Map<String, Map<String, List<Map<String, String>>>>>

      The problem is that when there is a running job there is a property called resourceRequests which has a json array as a value. Therefore Jackson throws an exception when it tries to deserialise a json array into a string.

      Proposed:

      Deserialise the json into the following structure: <Map<String, Map<String, List<Map<String, Object>>>>>. The only property being queried for is the `state` so I think this is appropiate.

      I have written the first test for this module. Very open to feedback and improvements.

      Attachments

        1. SAMZA-930.1.patch
          9 kB
          Yi Pan
        2. samza-930.patch
          8 kB
          Alex Buck

        Activity

          People

            alex.buck10 Alex Buck
            alex.buck10 Alex Buck
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: