Open-source intelligence (OSINT) is the practice of gathering information from public sources. It needs no privileged access, no vendor, and no budget. The uncomfortable part is that most of what it finds was published deliberately, by the target organisation itself.

We ran it against our own domain. Every query below targets something we operate, or a public log about us. That constraint isn't squeamishness. Reconnaissance against someone else's network without permission can be illegal, and a guide demonstrating it on a third party would be teaching the wrong habit. The technique transfers. The target should be yours.

What we volunteer without being asked

The fastest source is the site itself. One request, and the response headers said:

server: cloudflare
via: 1.1 Caddy

The first is unremarkable — a CDN, visible to anyone. The second is more interesting: it names the reverse proxy sitting behind the CDN. Nobody asked for that. It is a default, and it tells a stranger which software to look up when a vulnerability in it is next announced.

This is the shape of most OSINT findings. Not a leak, not a breach. A default nobody turned off.

The file that maps your interesting paths

Our robots.txt carries 26 Disallow rules. Among them: /admin/, /account/, /api/, /go/, /webhooks/ and /sudo.

Those rules exist for a good reason — they keep crawlers out of pages that should not be indexed, which is exactly what the file is for. But robots.txt is a public instruction to well-behaved crawlers, and it is the first thing anyone doing reconnaissance reads. A list of the paths you would rather nobody looked at is, from the other side of the table, a list of the paths worth looking at.

There is no clever fix here and we are not going to pretend otherwise. You cannot solve it by removing the rules — that just gets the pages indexed. The only response is to stop treating the file as a boundary. Everything it names must be secured as though it were linked from the homepage, because for an attacker, it is.

The rule of thumb: anything you would be uncomfortable seeing in a stranger's notes should not be the only thing standing between them and the endpoint. If /sudo is safe, naming it costs nothing. If naming it worries you, the problem is not the file.

Certificate transparency, and the ratchet nobody plans for

Every publicly-trusted TLS certificate issued anywhere is written to public, append-only, mirrored logs. This is a deliberate, load-bearing part of the web's security model for catching mis-issued certificates.

It also means that issuing a certificate publishes the hostname, permanently. A subdomain that was never linked from anywhere, never indexed, never mentioned — staging, dev, internal-api, an old customer's vanity name — enters a public record the moment a certificate covers it.

And it cannot be withdrawn. The logs are append-only and mirrored by multiple operators. Taking the host offline does not remove the record that it existed; it just means the name now resolves to nothing, which tells a reader that something used to be there.

Our own footprint is small: 2 certificates, 1 distinct name. That is not because we are careful about disclosure. It is because our staging environment uses an internal certificate authority, so its certificates never enter a public log at all.

That detail is more useful than a long list of subdomains would be, because it points to the control itself. If you do not want non-production hostnames in a permanent public record, do not issue publicly-trusted certificates for them. Use an internal CA, or a wildcard that names no individual host.

What DNS says about your suppliers

Our public DNS records sketch a map of our mail and DNS providers, none of it secret:

  • Mail exchangers point at Cloudflare Email Routing — so mail is not self-hosted, and a reader knows which provider's behaviour to expect
  • TXT records carry an SPF policy and a Google site-verification token
  • There is no CAA record, meaning we have not restricted which certificate authorities may issue for the domain

None of that is dangerous on its own. Collectively it sketches a supplier map, and supplier maps are how a competent attacker chooses which phishing pretext will look plausible. An invoice from a provider you actually use is a better lure than one from a provider you do not.

The CAA gap is the one we would fix first, and it is one DNS record.

Run it on yourself, in about ten minutes

Nothing here needs a tool you have to buy.

curl -sI https://yourdomain.com/ | grep -iE 'server|via|x-powered-by'
curl -s https://yourdomain.com/robots.txt
dig +short MX yourdomain.com
dig +short TXT yourdomain.com
dig +short CAA yourdomain.com

For certificate transparency, search a public log aggregator for your domain with subdomains included, and read the list of names as though you had never seen your own infrastructure before. The question is not "is this secret" — none of it is. The question is what does this list, taken together, tell someone about where to push.

Our DNS lookup tool runs the record queries from a browser if you would rather not use a terminal.

The defensive counterpart is three DNS records almost nobody publishes, which measures what the same class of institution does — and does not — publish to protect the people using it.

The honest limits

This was one domain, ours, at one moment. A larger organisation with more history will have a far bigger certificate transparency footprint, and that is where this technique produces its most uncomfortable findings — forgotten hosts from acquisitions, projects that were cancelled, environments somebody stood up in 2019.

Passive reconnaissance has a hard ceiling. It tells you what is published, not what is running or vulnerable. Everything above is reconnaissance. It stops short of touching your systems, which is why it's available to anyone, leaves no trace, and is almost always the first step.

Measured on 19 August 2026. The script is in our repository; we wrote it to fail loudly rather than report zero exposure if a source is unavailable, since an empty result here would be misleading.