[ROS 7.19] routeros_ip_service returns 'Bad Request this is configured elsewhere'

[yoo]

Describe the bug
I manage the ssl ceritficates for /ip/services with terraform. I configure the www-ssl services
with terraform while terraform is using the services to connect to the device.
This leads randomly to the error:

│ Error: POST 'https://<url>/rest/ip/service/set' returned response code: 400, message: 'Bad Request', details: 'failure: this is configured elsewhere'

The cause is that RouterOS 7.19 started to list all connections in /ip/services.
Changelog:

*) ip-service - show all TCP/UDP connections on the system;
*) ip-service - show all TCP/UDP ports on system, including ports in containers;
*) ip-service - show error message when service enable fails;

Now there are multiple "services" with the name/number www-ssl.

/ip/service/print
[...]
 9     www-ssl          443  tcp    letsencrypt.crt  main            20                                     
10 D c www-ssl          443  tcp                                                    <ip>    <ip>:43532
11 D c www-ssl          443  tcp                                                    <ip>    <ip>:47640
[...]

Entries with D c are dynamic connections. When the API request from terraform hits one of the dynamic connections instead of the right service the error is returned.

I found this forum therad: https://forum.mikrotik.com/viewtopic.php?p=1146407
Their solution is to filterer the set command with a sub query.

/ip/service set [find dynamic =no and name =ssh] port=2222 disabled=no

But I have no idea how this could be done through the rest API or inside the tf provider code.
If I could get some pointers on how to fix this I'm more than happy to open a PR.

To Reproduce

1. Enable the www service.

/ip/service set [find dynamic =no and name =www] port=80 disabled=no

2. Open the web interface of the Mikrotik device over http to create dynamic http connections in the /ip/service table.
3. Try to apply:

terraform {
  required_providers {
    routeros = {
      source  = "terraform-routeros/routeros"
      version = "1.85.3"
    }
  }
}

provider "routeros" {
  hosturl  = "http://<ip>:80"
  username = "[...]"
  password = "[...]"
}

resource "routeros_ip_service" "www" {
  numbers  = "www"
  port     = "80"
  disabled = false
}

This randomly results in:

Error: POST 'http://<ip>:80/rest/ip/service/set' returned response code: 400, message: 'Bad Request', details: 'failure: this is configured elsewhere'

[vaerh]

Hi!

I tried making a quick patch. It works, but it breaks backwards compatibility. I'll be thinking about what can be done about it.

[yoo]

Would you mind sharing the patch? I'd like to build the provider locally.

[vaerh]

@yoo
fixes here

[yoo]

Sadly the fix doesn't seem to work. The error persists. I played around with curl and the following requests all result in the same error:

# using url parameter like the proposed fix

curl -X POST -k -u "admin:${SEC}" https://${URL}/rest/ip/service/set\?dynamic\=false --data '{ "certificate":"letsencrypt.crt","disabled":"no","numbers":"www-ssl","port":"443","tls-version":"only-1.2"}' -H "content-type: application/json" 
{"detail":"failure: this is configured elsewhere","error":400,"message":"Bad Request"}

curl -X POST -k -u "admin:${SEC}" https://${URL}/rest/ip/service/set\?dynamic\=no --data '{ "certificate":"letsencrypt.crt","disabled":"no","numbers":"www-ssl","port":"443","tls-version":"only-1.2"}' -H "content-type: application/json"
{"detail":"failure: this is configured elsewhere","error":400,"message":"Bad Request"}

# using .query inside the json request data

curl -X POST -k -u "admin:${SEC}" https://${URL}/rest/ip/service/set --data '{ "certificate":"letsencrypt.crt","disabled":"no","numbers":"www-ssl","port":"443","tls-version":"only-1.2", ".query": ["dynamic=no", "#|"] }' -H "content-type: application/json"
{"detail":"failure: this is configured elsewhere","error":400,"message":"Bad Request"}

curl -X POST -k -u "admin:${SEC}" https://${URL}/rest/ip/service/set --data '{ "certificate":"letsencrypt.crt","disabled":"no","numbers":"www-ssl","port":"443","tls-version":"only-1.2", ".query": ["dynamic=false", "#|"] }' -H "content-type: application/json"
{"detail":"failure: this is configured elsewhere","error":400,"message":"Bad Request"}

curl -X POST -k -u "admin:${SEC}" https://${URL}/rest/ip/service/set --data '{ "certificate":"letsencrypt.crt","disabled":"no","numbers":"www-ssl","port":"443","tls-version":"only-1.2", ".query": ["dynamic=false"] }' -H "content-type: application/json" 
{"detail":"failure: this is configured elsewhere","error":400,"message":"Bad Request"}

curl -X POST -k -u "admin:${SEC}" https://${URL}/rest/ip/service/set --data '{ "certificate":"letsencrypt.crt","disabled":"no","numbers":"www-ssl","port":"443","tls-version":"only-1.2", ".query": ["dynamic=no"] }' -H "content-type: application/json"   
{"detail":"failure: this is configured elsewhere","error":400,"message":"Bad Request"}

From what I understand the patch fixes the read path but not the create/update path. This line seems to be the problem:

terraform-provider-routeros/routeros/resource_ip_service.go

Line 105 in d19886f

d.SetId(d.Get("numbers").(string))

[yoo]

Okay I figured it out. The problem is the numbers field in the request. Removing the numbers field and adding the correct .id fixes the issue.

curl -X POST -k -u "admin:${SEC}" https://${URL}/rest/ip/service/set --data '{ "certificate":"letsencrypt.crt","disabled":"no", "port":"443","tls-version":"only-1.2", ".id":"*6"}' -H "content-type: application/json"  

A very crude fix can be found here: yoo@7e47ce0

First read the .id from the ip service from the device. Than set the .id field on the request and remove the numbers field from the request.

[vaerh]

@yoo
I think this is a bug in 7.19 that was fixed in 7.19.1.

curl -sk -u "admin:${SEC}" https://rourer.local/rest/system/resource | jq .version
"7.19 (stable)"

curl -X POST -k -u "admin:${SEC}" -H "content-type: application/json" --data '{"certificate":"ssl","disabled":"no","numbers":"www-ssl","port":"443","tls-version":"only-1.2"}' https://router.local/rest/ip/service/set
{"detail":"failure: this is configured elsewhere","error":400,"message":"Bad Request"}

curl -sk -u "admin:${SEC}" https://rourer.local/rest/system/resource | jq .version
"7.19.1 (stable)"

curl -X POST -k -u "admin:${SEC}" -H "content-type: application/json" --data '{"certificate":"ssl","disabled":"no","numbers":"www-ssl","port":"443","tls-version":"only-1.2"}' https://router.local/rest/ip/service/set
[]
curl -k -u "admin:${SEC}" "https://router.local/rest/ip/service?name=www-ssl&dynamic=no"
[{".id":"*6","address":"","certificate":"ssl","disabled":"false","dynamic":"false","invalid":"false","max-sessions":"20","name":"www-ssl","port":"443","proto":"tcp","tls-version":"only-1.2","vrf":"main"}]

I have once again made changes to the provider as it works for versions 7.15 and 7.19.1. And I will probably prepare the release.