Don't require EIP for HA mode

[JulianCBC]

This ports most of int128/terraform-aws-nat-instance#52 to fck-nat and builds upon #11.

The only requirement for NAT to work is a functional internet connection, so as the NAT EC2 instance is required to be running on a public subnet, we don't actually need a EIP to get a public IP and therefore an internet connection. (Also they're a very limited resource to be required by a "cheap" NAT solution)

Update the scripting to use eth0 for the upstream internet connection instead of deconfiguring it.

This is untested, but is functionally identical to the referenced pull request in the Terraform module which works in my testing. Note that I have not tested port forwarding at all, so it's possible that these changes will break that, however it's unlikely as Linux's routing generally doesn't care where packets come from.

Note that this set of changes will break any system that expects to have an open port on the same IP as it sends from. The fix is to update the port forwarding rules to expect packets on eth0 instead of eth1.

[AndrewGuenther]

This will not work as written. For traffic to come out of eth0 you'll need routing rules to bridge incoming traffic from eth1 to go out eth0. That's the ideal implementation of HA mode, but I was having problems bridging the interfaces in AL2 so I went with the quick-and-dirty EIP implementation initially instead.

[JulianCBC]

For traffic to come out of eth0 you'll need routing rules to bridge incoming traffic from eth1 to go out eth0.

You don't, routing in Linux is essentially one big table with all input traffic from all sources being dealt with identically.

MASQUERADE is effectively a rule that says "ok, now this traffic is from me", so as long as the traffic gets to that rule, whatever happens next doesn't matter as long as there's some way to get out.

NAT instances like this will always have an internet connection on eth0, so we might as well use it.

That's the ideal implementation of HA mode, but I was having problems bridging the interfaces in AL2 so I went with the quick-and-dirty EIP implementation initially instead.

That's what it looked like to me when testing this, but the issue was that the reverse path filter was silently dropping the traffic. I've got a functionally identical version of these changes passing traffic perfectly right now.

[AndrewGuenther]

You don't, routing in Linux is essentially one big table with all input traffic from all sources being dealt with identically.

MASQUERADE is effectively a rule that says "ok, now this traffic is from me", so as long as the traffic gets to that rule, whatever happens next doesn't matter as long as there's some way to get out.

That's what I had thought, but I had been having issues with it. I've implemented this all before, and never remember having to bridge anything...

That's what it looked like to me when testing this, but the issue was that the reverse path filter was silently dropping the traffic.

That'll do it. 🤦‍♂️

Alright, there's some other components which will need updating for this. I'll test this end to end and add those changes here.

[JulianCBC]

Yeah, getting this working has been almost 5 days of frustration for me, so I'm happy that there's a resolution and something I can contribute.

Alright, there's some other components which will need updating for this. I'll test this end to end and add those changes here.

Go ahead, I don't really mind what happens to these changes as long as they get out there =)

[AndrewGuenther]

Deployed this in a test stack and it isn't working out of the box. I'll need to do some more digging another day, but if it is working in terraform-aws-nat-instance there's no reason it shouldn't work here.

[AndrewGuenther]

Alright, I tried one more thing and it turns out you do need src/dest checks disabled on eth0. I've got a script working that could use some cleanup that I can push up tomorrow.

[JulianCBC]

This has a not-entirely-hack solution to applying this to everything here: https://github.com/int128/terraform-aws-nat-instance/pull/51/files

Probably not the best solution, but it works 😅

[AndrewGuenther]

Pushed my version of this up to !13. I want to rename some of the config options, but otherwise this is a great improvement overall. Thanks so much!

[JulianCBC]

You're most welcome, I'm closing this pull request now as everything I've done is in your new one.