Dead Domain Detection Methods?

[ghost]

Hi,

Just wondering what detection methods do you use and have planed for dead domains?

E.g.

• DNS NS Record
• DNS A/AAAA Record
• DeadDomainsLinter - urlfilter.adtidy.org
• WHOIS
• RDAP
• etc

[funilrys]

Hey @bxn8 we use https://github.com/funilrys/PyFunceble . I hope it answers your question.

If you have some ideas on a new method, feel free to propose them. I'm open for improvements.

[ghost]

@funilrys Thanks for getting back to me.

What are your current command parameters for PyFunceble? For example, the default -d command checks the HTTP status, which is fairly inaccurate for domain lists that focus on malware rather than ads, as they tend to block random HTTP requests to the malicious server.

Also, do you store a central cache file of dead domains to improve processing times?

[funilrys]

Hey @bxn8,

If Pyfunceble shows HTTP Status, it doesn't necessarily mean that it only checks the HTTP status.

PyFunceble works as follows:

WHOIS -> If not successful -> DNS -> if not successful -> HTTP -> and so on until there is no testing method anymore.

Each of the testing methods can be disabled. But PyFunceble go through all authorized methods.

As for the infrastructure here, we have our own software: https://github.com/dead-hosts/dev-center

And our own configuration: https://github.com/dead-hosts/infrastructure-launcher/blob/master/dead_hosts/launcher/defaults/pyfunceble.py#L47-L71

Which can me overwritten by anyone through the info.json: https://github.com/dead-hosts/repository-structure/blob/83a0cf75b8bce02c2baeb5a3570b34a018c97a76/info.json#L3

I'm working on a centralized platform to share and distribute workloads across multiple decentralized nodes. It's not scaling yet, but I can invite you if you want. Feel free to reach out on Discord (fast) or via Email (slow).

[ghost]

@funilrys Great to hear, will reach out via email.