Packages Cannot Be Installed When Yum Transactions Fail

Many installations are running into an issue with installing new bits in preparation for an upgrade. Consider the following stack trace:

2017-11-01 18:52:40,456 - No package found for storm_${stack_version}(storm_(\d|_)+$) 
2017-11-01 18:52:40,457 - PackageNone
{'retry_on_repo_unavailability': False, 'retry_count': 5, 'action': ['upgrade']}

2017-11-01 18:52:40,457 - Installing package None ('/usr/bin/yum -d 0 -e 0 -y install ''') 
2017-11-01 18:52:41,308 - Execution of '/usr/bin/yum -d 0 -e 0 -y install ''' returned 1. Error: Nothing to do

Ambari attempts to determine the correct package to install by first doing a scoped search by a specific repository. Once it has found a match, it then proceeds with the yum install.

The problem in the above system is that Storm appears to have already been partially installed:

[root@c6401 ~]# yum list installed | grep storm
storm_2_6_0_0_334.x86_64    0.7.0.2.6.0.0-334.el6    installed

Notice that the repository listed for storm shows installed, instead of a real repository. The actual repository it should be coming from is called HDP-2.6-repo-1.

The odd part here is that this appears to be our first install of storm. If we can't find the package, then how did it even get into this state? It seems like there might be something going on in the agent in terms of killing yum and automatically retrying it. We can see the following:

[root@c6401 ~]# yum-complete-transaction
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.solarvps.com
 * extras: mirrors.mit.edu
 * updates: mirror.5ninesolutions.com
There are 1 outstanding transactions to complete. Finishing the most recent one
The remaining transaction had 1 elements left to run
--> Running transaction check
---> Package storm_2_6_0_0_334.x86_64 0:1.1.0.2.6.0.0-334.el6 will be installed
--> Finished Dependency Resolution

It's actually pretty easy to duplicate this problem - during a yum install, just kill yum. It will leave the package in this quasi-installed state where its repo is listed as 'installed' even though it is not.

I think what's happening here is that the agent receives the command to install storm. During the course of the installation - the yum command gets killed (possibly by the agent itself) and the agent retries to install the package quietly. It then is unable to find the package anymore since it no longer is listed as "available" and is now "installed" with the installed repository association.