Credentials issue

[mphilip9]

Logstash version: 7.17.9

Error logged when running logstash (sudo systemctl start logstash):

Plugin: <LogStash::Inputs::CloudWatch_Logs access_key_id=>"**", secret_access_key=><password>, id=>"68ef61a27eed2bd3e2a969415fd27263924ca794001390c75f15c212eab", log_group=>["fluteui-Prod", "

logstash[30989]: Error: The security token included in the request is invalid.
 Exception: Aws::CloudWatchLogs::Errors::UnrecognizedClientException

The access key is passed correctly, but the secret_key is a somehow just password. During setup, the secret key and the access key are added to the cloudwatch_logs_cloud.conf file:

sudo sed -i "s/access_here/$(sed 's:/:\\/:g' /home/access_key_id.txt)/" /etc/logstash/conf.d/cloudwatch_logs_cloud.conf
sudo sed -i "s/secret_here/$(sed 's:/:\\/:g' /home/secret_access_key.txt)/" /etc/logstash/conf.d/cloudwatch_logs_cloud.conf

I have looked at both .txt files and the cloudwatch.conf file and the keys are there, but somehow the secret key is being passed incorrectly. Has anyone encountered a problem like this before?

Here is the cloudwatch.conf file. You can see I tried using an aws_credentials.yaml file as well, and I got another invalid token error:

input {	
   cloudwatch_logs {
 #       access_key_id => **
 #       secret_access_key => **
	    aws_credentials_file => "/etc/logstash/aws_credentials.yaml"
        log_group => [ "xwingui-Prod", "xwingui-Dev", "xwingui-Exp", "xwingui-Staging", "xwingui-Test", "xwingui-Jawn"  ]
        region => "us-east-1"
        sincedb_path => "/var/lib/.sincedb"
    }
}

filter {
    if "Monitoring - " in [message] {
        if "API" in [message] {
            grok {
                match => { "message" => "API Monitoring - %{GREEDYDATA:json}" }
            }
            mutate {
                add_field => { "monitorType" => "API" }
            }
        } else if "RUM" in [message] {
            grok {
                match => { "message" => "RUM Monitoring - %{GREEDYDATA:json}" }
            }
            mutate {
                add_field => { "monitorType" => "RUM" }
            }
        } else if "PikaWorker" in [message] {
            grok {
                match => { "message" => "PikaWorker Monitoring - %{GREEDYDATA:json}" }
            }
            mutate {
                add_field => { "monitorType" => "PikaWorker" }
            }
        } else if "DataAgent" in [message] {
            grok {
                match => { "message" => "DataAgent Monitoring - %{GREEDYDATA:json}" }
            }
            mutate {
                add_field => { "monitorType" => "DataAgent" }
            }
        } else if "Database" in [message] {
            grok {
                match => { "message" => "Database Monitoring - %{GREEDYDATA:json}" }
            }
            mutate {
                add_field => { "monitorType" => "Database" }
            }
        } 

        json {
            source => "json"
            remove_field => "message"
        }
        mutate {
            add_field => { "isMonitor" => True }
        }
    }
}

output {
    elasticsearch {
        hosts => [ "localhost:9200" ]
        user => user_here
        password => **
    }
    stdout {
        codec => json
    }
}

[koshkin-ccna]

I really hope that you have not posted your real aws credentials here.

[mphilip9]

They are dummy credentials

[sister-bael]

Is this repo dead? not any recent activity from maintainer and this issue is blocking me