Accuracy depends on the question
A result can be correct at country level and wrong at city level. Evaluating “accuracy” therefore requires defining the field, the acceptable distance/error, the network population and known ground truth.
Provider agreement is useful but not ground truth
If two independent providers return the same country and city, that is evidence of consistency. It does not prove that the device is physically there. Our Location Agreement metric is intentionally named to preserve that distinction.
Network type changes the challenge
Residential broadband, mobile carrier networks, VPN exits, cloud platforms, satellite services and enterprise gateways create very different mapping problems. A benchmark dominated by datacenter IPs cannot automatically describe mobile-user accuracy.
How we would benchmark responsibly
- Use a documented sample with independently known ground truth.
- Separate country, region and city metrics.
- Report error distance distributions, not only averages.
- Separate network types and IPv4/IPv6.
- Record collection dates and provider versions.
- Publish methodology so the test can be reproduced.
How to judge one lookup today
Compare sources, inspect the ASN/ISP, look at coordinate distance, and treat a city disagreement as uncertainty. If an application requires physical presence, use an appropriate location mechanism rather than IP alone.
Why accuracy is not one number
A provider can be very good at identifying the correct country while still missing the city by tens or hundreds of kilometers. Accuracy also changes by ISP, region, mobile versus fixed network, IPv4 versus IPv6 and how recently an address range was reassigned.
How to judge one lookup responsibly
Look for agreement between providers, inspect the ASN and network type, and treat coordinates as representative rather than device-level. If a decision requires precise location, use a location method designed for that purpose instead of stretching IP data beyond its limits.