6 Commits

Author	SHA1	Message	Date
Dan Milne	a7291bdfc3	add test to ensure allowed clause with value also works	2024-12-24 10:44:06 +11:00
Dan Milne	88c7dc67f2	Add code and tests to handle a disallowed clause with value	2024-12-24 10:43:17 +11:00
Dan Milne	c700c09021	Bump the version	2023-09-10 13:18:11 +10:00
Dan Milne	71bbd4d1ad	Support for mulitple sitemaps. Sitemaps should be absolute, so we'll try and make them absolute - but if we're passed the Robots.txt as text we can't determine the host.	2023-09-10 13:17:53 +10:00
Dan Milne	27a47700e1	Gemspec adds the version, but tests also need it.	2023-09-10 10:31:46 +10:00
Dan Milne	5774ca2836	Switch to Net::HTTP and set the request header to match the query header	2023-09-10 10:02:52 +10:00