Performance Regressions in streaming python ParDo and GBK Load Tests

As specified in the Beam Release Guide, I'm investigating performance regressions. The following load test metrics show a clear and persistant performance regression starting approximately around March 17 and affecting version 2.38.0.

ParDo Load Tests: http://metrics.beam.apache.org/d/MOi-kf3Zk/pardo-load-tests?orgId=1&var-processingType=streaming&var-sdk=python
GBK Load Tests: http://metrics.beam.apache.org/d/UYZ-oJ3Zk/gbk-load-tests?orgId=1&var-processingType=streaming&var-sdk=python&from=now-30d&to=now