hosting.config not matching with dynamic regex remap

See https://github.com/Comcast/traffic_control/issues/1562

The hosting.config has an entry for the origin created in the Delivery service UI for a live server (volume=2 which is RAM). The problem comes when we use a REGEX to set an origin dynamically.

Example Origin URL : http://server01.kabletown.net

Then the Regex is the following : ^/(server\w\d\d)/([^?]+)(?:\.m3u8|)(.*)$ http://$1-.kabletown.net/$2

The result in the hosting.config is :

hostname=server01.kabletown.net volume=2
hostname=* volume=1
If the remap ends up with server02.kabletown.net then disks are used instead of RAM.
@smalenfant smalenfant added bug Traffic Ops labels on Jun 28, 2016
@smalenfant
Member
smalenfant commented on Jun 28, 2016
There is also another side effect with the parent.config.
@knutsel
Member
knutsel commented on Jun 29, 2016
We could add hosting_override and parent_override columns to the delveryservice table, but that seems a bit too ATS specific? Or an overrides table that is linked to the ds table that has file -> override line?

@dg4prez : you had similar use cases recently, I think? Any thoughts on this?
@dg4prez
Collaborator
dg4prez commented on Jun 29, 2016 • edited
Perhaps rather than cluttering up regex remap with more overrides for that specific use case, we remove regex remap in favor of adding more manual options to any_map. Then we could directly write the configs entries we want. I've given some rough thought to how we'd do that, though I haven't applied it to the hosting.config so we'd need to add that in there as well. Note that these would all be for advanced users who fully comprehend how traffic server works. In my opinion this is already the case for any_map and regex remap, anyway.

Database changes

add table anymap (deliveryservice_anymap)
ID, remap text, edge_header, mid_header, type (edge, mid, etc), last updated

add table anyparent (deliveryservice_edgeparent)
ID, parent text, type (edge, mid, etc), last updated
#autogenerated by cachegroup for edge types, fully manual for mid

add table includes (deliveryservice_includes)
ID, file_name, file_content (this would need to be a large field), type (edge, mid, etc), last updated

UI

Origin server base URL - remove this
Use MSO feature - remove this

Remap Entries - create entries the same way we create them for regex entries - add one or a whole bunch. probably don't need order. Group of the following for each.

Raw Remap Text:
Edge Header Rewrite:
Mid Header Rewrite:
Tier Assignment: (edge, mid, etc checkboxes)

Note that in order to include header rewrite entries for each the header name would need to be something like xml_id_1, xml_id_2, etc. Not sure of the best answer, yet. Maybe the best way would be to place a line under each header saying what the user would need to add to the remap line in order to include it, or maybe the best way would be to have the system automatically append it.

Parent entries - Same as remap entries, add as many as needed

Mid Parent text:
Tier assignment: (edge, mid, etc checkboxes)

Manual includes - files we want to reference on the map lines like header rewrites etc that don't exist. We'd add each one, and the user would include the filename and contents.