{"id":90942,"date":"2026-09-15T09:50:28","date_gmt":"2026-09-15T07:50:28","guid":{"rendered":"https:\/\/www.humanlevel.com\/?p=90942"},"modified":"2026-09-15T09:50:28","modified_gmt":"2026-09-15T07:50:28","slug":"block-bots-negotiate-access-management-discoverability","status":"publish","type":"post","link":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability","title":{"rendered":"From blocking bots to negotiating access: the new approach to findability"},"content":{"rendered":"\n<p>Yesterday we learned that <a href=\"https:\/\/digiday.com\/media\/google-rolls-out-pay-value-ai-licensing-program-to-publishers\/\"><strong>Google has been quietly paying dozens of publishers for months<\/strong><\/a> when their content contributes significantly to its AI answers. This isn\u2019t an isolated move, but <strong>the latest piece that defines a changing of the guard in the relationship between AI bots and our content<\/strong>. But let\u2019s take it step by step.  <\/p>\n\n\n\n<p>Not all AI bots that access our website are after the same thing. And starting to tell them apart is becoming increasingly important for our business. <\/p>\n\n\n\n<p>Until now, it\u2019s been common to talk about AI bots as if they were a single category. Basically, most websites either let them in or block them, and that\u2019s that. <\/p>\n\n\n\n<p>But <strong>something has started to change<\/strong>.<\/p>\n\n\n\n<p><a href=\"https:\/\/www.akamai.com\/blog\/security\/introducing-more-granular-controls-ai-bot-traffic\">Just a few days ago, Akamai announced it has updated its bot directory to differentiate AI traffic<\/a> into three broad groups:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Training bots<\/strong>: those that collect content to train AI models.<\/li>\n\n\n\n<li><strong>Search bots<\/strong>: those that crawl and feed the platform\u2019s own results index so that content can be cited and linked in its generative answers.<\/li>\n\n\n\n<li><strong>AI agents<\/strong>: those that access a website in real time because a user has asked them to look up information, compare products, or carry out a task.<\/li>\n<\/ul>\n\n\n\n<p>While the differences may seem like minor technical nuances, <strong>the consequences have a very clear impact<\/strong>. Let\u2019s see. <\/p>\n\n\n\n<p>We might, for example, <strong>not want some of our content<\/strong> (for instance, informational pages, FAQs, studies, analyses, etc.) <strong>to be used to train models, but still want it to appear as a source<\/strong> when someone asks ChatGPT or Perplexity about our products or services. And we probably <strong>don\u2019t want to block an agent that a customer has asked to consult our catalog<\/strong>, look up a price, compare a solution, or perform an action on our website, either. <\/p>\n\n\n\n<p><strong>Akamai therefore enables much more granular control<\/strong> so website owners can <strong>decide when and how their content is accessible<\/strong>.<\/p>\n\n\n\n<p>What\u2019s most striking is the <strong>timing of Akamai\u2019s decision alongside a series of changes that suggest the era in which LLMs accessed all online content without permission or control is coming to an end<\/strong>.<\/p>\n\n\n\n<p>In recent months, <strong>a regulator has turned it into a right<\/strong>, <strong>major media groups are steering it toward their bottom line, the web\u2019s own infrastructure has adopted it as a default setting, and even Google itself<\/strong>, as we saw at the start, <strong>has begun paying for the content<\/strong> that feeds its AI answers.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The regulator turns the distinction into a right<\/h2>\n\n\n\n<p>The Competition and Markets Authority (CMA), the UK competition regulator, issued on June 3 <a href=\"https:\/\/www.gov.uk\/government\/news\/cma-secures-fairer-deal-for-publishers-and-improves-google-search-services-in-uk\"><strong>a binding order<\/strong> (the first of its kind in the world) requiring Google to offer publishers separate opt-out controls for three areas of its results<\/a>: AI Overviews, conversational AI Mode, and <em>fine-tuning<\/em> of its AI models, such as Gemini. The real novelty of this order is that it requires the search engine to ensure that a publisher\u2019s adoption of any of these opt-outs <strong>does not penalize its rankings in traditional search results<\/strong>. <\/p>\n\n\n\n<p>Until then, <strong>Google forced an all-or-nothing choice<\/strong>: either your content fed all its AI features or, <a href=\"https:\/\/seranking.com\/blog\/seo-news-how-to-hide-ai-overviews\/\">if you implemented the relevant block in robots.txt, you disappeared from the search engine<\/a>. The CMA has put an end to this model, and control can now be exercised at the domain level or on individual pages. In other words, <strong>you can let your evergreen content section appear in AI Overviews while protecting your premium or subscription content from Gemini fine-tuning<\/strong>. In addition, Google must attribute\u2014with clear links\u2014the content it uses in its AI answers, and the order expressly prohibits Google from hiring crawlers from other companies to access, through the back door, the content of those who have chosen to block its access.   <\/p>\n\n\n\n<p>The UK regulator\u2019s order states that <strong>Google has nine months to implement the changes<\/strong> and, for now, the features are being rolled out only to a subset of UK sites. But we can see <strong>a clear pattern: what Akamai treats as technical categories, the CMA enshrines as separate rights for the content owner<\/strong>. And where the UK has led the way with its Digital Markets, Competition and Consumers Act, it\u2019s reasonable to expect other regulators, such as the European Commission, to take note.  <\/p>\n\n\n\n<img loading=\"lazy\" width=\"1500\" height=\"780\" class=\"gb-media-e0ff413f\" alt=\"\" title=\"uk-cma-control\" src=\"https:\/\/www.humanlevel.com\/wp-content\/uploads\/uk-cma-control.jpg\" srcset=\"https:\/\/www.humanlevel.com\/wp-content\/uploads\/uk-cma-control.jpg 1500w, https:\/\/www.humanlevel.com\/wp-content\/uploads\/uk-cma-control-754x392.jpg 754w, https:\/\/www.humanlevel.com\/wp-content\/uploads\/uk-cma-control-1200x624.jpg 1200w, https:\/\/www.humanlevel.com\/wp-content\/uploads\/uk-cma-control-400x208.jpg 400w, https:\/\/www.humanlevel.com\/wp-content\/uploads\/uk-cma-control-900x468.jpg 900w\" sizes=\"auto, (max-width: 1500px) 100vw, 1500px\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">Media groups use this distinction to boost their bottom line<\/h2>\n\n\n\n<p>Without waiting for regulators, major publishing groups and international <em>publishers<\/em> are already applying the same logic. The pattern repeats: <strong>on the one hand, we have <a href=\"https:\/\/digiday.com\/media\/a-timeline-of-the-major-deals-between-publishers-and-ai-tech-companies-in-2025\/\">media outlets signing licensing deals with certain AI companies<\/a> and, at the same time, <a href=\"https:\/\/www.searchenginejournal.com\/more-news-sites-default-to-blocking-ai-crawlers\/578527\/\">blocking access for those that don\u2019t pay<\/a><\/strong>. <strong>LLM access to content<\/strong> stops being free and universal and <strong>becomes a negotiable asset<\/strong>. <\/p>\n\n\n\n<p>The most sophisticated case is The Atlantic. <a href=\"https:\/\/digiday.com\/media\/inside-the-atlantics-ai-bot-blocking-strategy\/\">As Digiday has reported<\/a>, <strong>the magazine\u2019s team tracks in a spreadsheet which <em>crawlers<\/em> access its site and cross-references that data with referral traffic and subscription conversions generated by each AI platform<\/strong>. Each week they review bot behavior and decide which ones get access and which don\u2019t. That analysis led them, for example, to <a href=\"https:\/\/x.com\/nxthompson\/status\/1990499169111523455?s=20\">block a single <em>crawler<\/em> that had made 564,000 requests in seven days without giving anything back in return<\/a>. In this way, <strong>The Atlantic blocks only the bots whose cost exceeds the value they deliver<\/strong>.   <\/p>\n\n\n\n<p>And they\u2019re not alone. <a href=\"https:\/\/digiday.com\/media\/reuters-and-time-adopt-bot-blocking-whitelists-to-rein-in-ai-crawlers\/\">Reuters and Time have also moved to blocking AI bots by default<\/a>, allowing access only to those approved via allowlists. Their teams say the change not only didn\u2019t cost them traffic, but also significantly reduced their server spend\u2014and that it has also helped <strong>encourage AI companies to come to the negotiating table<\/strong>. Selective blocking isn\u2019t just a defensive measure; it becomes a <strong>commercial lever<\/strong>.  <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Google puts a price on the other side of the scale<\/h2>\n\n\n\n<p>And while publishers figure out how to value each bot, <strong>Google has started to value each piece of content<\/strong>. As Digiday reported yesterday, <strong>the company has been quietly testing the <a href=\"https:\/\/digiday.com\/media\/google-rolls-out-pay-value-ai-licensing-program-to-publishers\/\">&#8220;<em>AI contribution pilot&#8221;<\/em><\/a>: a program built into Search Console that pays publishers when their content contributes &#8220;significantly&#8221; to AI-generated answers<\/strong> in Gemini, AI Overviews, and AI Mode. Publishers participating in this pilot program have an AI revenue <em>widget<\/em> in their Google Search Console dashboard showing a monthly figure. <strong>Payment doesn\u2019t depend on crawl volume, but on the value contributed<\/strong> by the content. In other words, Google only pays when it judges that a piece has made a meaningful contribution to an answer.   <\/p>\n\n\n\n<p>While it\u2019s a step in the right direction, <a href=\"https:\/\/www.searchenginejournal.com\/google-tests-paying-publishers-for-ai-answers-via-search-console\/589414\/\">participants themselves describe the calculation as a black box<\/a>, the revenue level is negligible compared to their ad income, and <strong>some analysts interpret the program as a test run of the infrastructure Google would need if it were ever forced to pay for the content it uses<\/strong>. The program has also been more attractive to small and mid-sized publishers\u2014who lack the muscle to negotiate their own licensing deal\u2014than to the largest <em>publishers<\/em>. <\/p>\n\n\n\n<p>But <strong>the conceptual shift is important: through this data, publishers will be able to see which content delivers value to AI systems<\/strong> in the same way Search Console shows us today which content performs in search. This establishes that <strong>some content does, in fact, deliver value\u2014and there must be a path for that value to flow back to the people who created it<\/strong>. Exactly the same logic The Atlantic applies to bots, but in reverse.  <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Infrastructure, set by default<\/h2>\n\n\n\n<p>In case there were any doubts that <strong>this is the new standard<\/strong> and not a fad: <strong>starting today, September 15, <a href=\"https:\/\/developers.cloudflare.com\/bots\/additional-configurations\/block-ai-bots\/\">new domains signing up to Cloudflare will have training <em>crawlers<\/em> and agents blocked by default on pages that serve ads<\/a><\/strong>, while search bots remain allowed. When companies as large as Akamai or Cloudflare\u2014which underpin a large share of the web\u2014adopt the taxonomy as the default server configuration, the debate is settled: <strong>granular management of AI bot access stops being a novelty and becomes the ground for negotiation<\/strong>. <\/p>\n\n\n\n<img loading=\"lazy\" width=\"1500\" height=\"780\" class=\"gb-media-b8232070\" alt=\"\" title=\"bloqueo-contenidos\" src=\"https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-contenidos.jpg\" srcset=\"https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-contenidos.jpg 1500w, https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-contenidos-754x392.jpg 754w, https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-contenidos-1200x624.jpg 1200w, https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-contenidos-400x208.jpg 400w, https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-contenidos-900x468.jpg 900w\" sizes=\"auto, (max-width: 1500px) 100vw, 1500px\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">From \u201csomething for everyone\u201d to selective tolls<\/h2>\n\n\n\n<p><strong>Five different players<\/strong>\u2014an infrastructure provider, a regulator, major media groups, the world\u2019s leading CDN, and Google itself\u2014<strong>have converged in just a few months on the same idea: AI access to content has a value that must be measured, negotiated, and increasingly, paid for.<\/strong><\/p>\n\n\n\n<p><strong>It\u2019s time to change how we think: in the face of the growing zero-click phenomenon<\/strong>, <strong>the question we should be asking<\/strong> is no longer whether to allow or block AI bots, but <strong>what type of access we allow in each case\u2014and in exchange for what value.<\/strong> <strong>Protecting our content<\/strong> and, at the same time, <strong>pursuing visibility<\/strong> in generative answers <strong>are no longer incompatible goals<\/strong>.<\/p>\n\n\n\n<p>One last note: managing all this takes more than editing robots.txt. According to the <a href=\"https:\/\/tollbit.com\/state-of-the-bots\/\">Tollbit State of the Bots report<\/a>,  <strong>around 30% of AI bot crawls fail to comply with the explicit permissions declared in that file. Bot governance requires log monitoring, CDN or WAF rules, and a periodic review of the value each bot delivers <\/strong> in exchange for the server resources it consumes.<\/p>\n\n\n\n<p>And <strong>as AI agents become increasingly widespread for search, comparison, or purchasing tasks, this distinction will undoubtedly become even more important<\/strong>: blocking the wrong agent will increasingly mean shutting the door on the potential customer who sent it.<\/p>\n\n\n\n<p>Our recommendation? Start with the basics: <strong>audit which bots access your site today, measure what each one gives you back, and decide\u2014bot by bot\u2014who gets in, under what terms, and at what price<\/strong>. <\/p>\n","protected":false},"excerpt":{"rendered":"<p>Learn how regulators, publishers, and CDNs are turning AI access to your content into negotiable value.<\/p>\n","protected":false},"author":3,"featured_media":90941,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[349],"tags":[],"class_list":["post-90942","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-seo"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>From blocking bots to negotiating access: manage your findability | Human Level<\/title>\n<meta name=\"description\" content=\"Learn how regulators, publishers, and CDNs are turning AI access to your content into negotiable value.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"From blocking bots to negotiating access: the new approach to findability\" \/>\n<meta property=\"og:description\" content=\"Learn how regulators, publishers, and CDNs are turning AI access to your content into negotiable value.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability\" \/>\n<meta property=\"og:site_name\" content=\"Human Level\" \/>\n<meta property=\"article:published_time\" content=\"2026-09-15T07:50:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.humanlevel.com\/wp-content\/uploads\/blocking-bots-social.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"675\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Fernando Maci\u00e1\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"From blocking bots to negotiating access: the new approach to findability\" \/>\n<meta name=\"twitter:description\" content=\"Learn how regulators, publishers, and CDNs are turning AI access to your content into negotiable value.\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.humanlevel.com\/wp-content\/uploads\/blocking-bots-social.jpg\" \/>\n<meta name=\"twitter:creator\" content=\"@fernandomacia\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Fernando Maci\u00e1\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":[\"Article\",\"BlogPosting\"],\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability\"},\"author\":{\"name\":\"Fernando Maci\u00e1\",\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en#\\\/schema\\\/person\\\/e85eb7565a8e3e5e682084fac681d8df\"},\"headline\":\"From blocking bots to negotiating access: the new approach to findability\",\"datePublished\":\"2026-09-15T07:50:28+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability\"},\"wordCount\":1592,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.humanlevel.com\\\/wp-content\\\/uploads\\\/bloqueo-bots.jpg\",\"articleSection\":[\"SEO\\\/GEO\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability\",\"url\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability\",\"name\":\"From blocking bots to negotiating access: manage your findability | Human Level\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.humanlevel.com\\\/wp-content\\\/uploads\\\/bloqueo-bots.jpg\",\"datePublished\":\"2026-09-15T07:50:28+00:00\",\"description\":\"Learn how regulators, publishers, and CDNs are turning AI access to your content into negotiable value.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability#primaryimage\",\"url\":\"https:\\\/\\\/www.humanlevel.com\\\/wp-content\\\/uploads\\\/bloqueo-bots.jpg\",\"contentUrl\":\"https:\\\/\\\/www.humanlevel.com\\\/wp-content\\\/uploads\\\/bloqueo-bots.jpg\",\"width\":272,\"height\":272,\"caption\":\"From blocking bots to negotiating access\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\\\/block-bots-negotiate-access-management-discoverability#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Inicio\",\"item\":\"https:\\\/\\\/www.humanlevel.com\\\/en\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Blog\",\"item\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"SEO\\\/GEO\",\"item\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/blog\\\/seo\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"From blocking bots to negotiating access: the new approach to findability\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en#website\",\"url\":\"https:\\\/\\\/www.humanlevel.com\\\/en\",\"name\":\"Human Level\",\"description\":\"Web positioning and online marketing consultant Human Level\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.humanlevel.com\\\/en?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en#organization\",\"name\":\"Human Level\",\"url\":\"https:\\\/\\\/www.humanlevel.com\\\/en\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.humanlevel.com\\\/wp-content\\\/uploads\\\/logohl25x3.png\",\"contentUrl\":\"https:\\\/\\\/www.humanlevel.com\\\/wp-content\\\/uploads\\\/logohl25x3.png\",\"width\":600,\"height\":93,\"caption\":\"Human Level\"},\"image\":{\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.linkedin.com\\\/company\\\/human-level-communications\",\"https:\\\/\\\/www.youtube.com\\\/user\\\/humanlevelcommunica\",\"https:\\\/\\\/bsky.app\\\/profile\\\/humanlevel.bsky.social\",\"https:\\\/\\\/instagram.com\\\/humanlevel\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/en#\\\/schema\\\/person\\\/e85eb7565a8e3e5e682084fac681d8df\",\"name\":\"Fernando Maci\u00e1\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.humanlevel.com\\\/wp-content\\\/uploads\\\/1x1-fernando-macia-26-2-96x96.jpg\",\"url\":\"https:\\\/\\\/www.humanlevel.com\\\/wp-content\\\/uploads\\\/1x1-fernando-macia-26-2-96x96.jpg\",\"contentUrl\":\"https:\\\/\\\/www.humanlevel.com\\\/wp-content\\\/uploads\\\/1x1-fernando-macia-26-2-96x96.jpg\",\"caption\":\"Fernando Maci\u00e1\"},\"description\":\"Founder and CEO of Human Level. Expert SEO consultant with more than 20 years of experience. He has been a professor at numerous universities and business schools, and director of the Master in Professional SEO and SEM and the Advanced SEO Course at KSchool. Author of a dozen books on SEO and digital marketing.\",\"sameAs\":[\"https:\\\/\\\/www.amazon.es\\\/Fernando-Macia-Domene\\\/e\\\/B004G45KK6\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/in\\\/fernandomacia\\\/\",\"https:\\\/\\\/x.com\\\/fernandomacia\"],\"url\":\"https:\\\/\\\/www.humanlevel.com\\\/en\\\/author\\\/fernando\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"From blocking bots to negotiating access: manage your findability | Human Level","description":"Learn how regulators, publishers, and CDNs are turning AI access to your content into negotiable value.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability","og_locale":"en_US","og_type":"article","og_title":"From blocking bots to negotiating access: the new approach to findability","og_description":"Learn how regulators, publishers, and CDNs are turning AI access to your content into negotiable value.","og_url":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability","og_site_name":"Human Level","article_published_time":"2026-09-15T07:50:28+00:00","og_image":[{"width":1200,"height":675,"url":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/blocking-bots-social.jpg","type":"image\/jpeg"}],"author":"Fernando Maci\u00e1","twitter_card":"summary_large_image","twitter_title":"From blocking bots to negotiating access: the new approach to findability","twitter_description":"Learn how regulators, publishers, and CDNs are turning AI access to your content into negotiable value.","twitter_image":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/blocking-bots-social.jpg","twitter_creator":"@fernandomacia","twitter_misc":{"Written by":"Fernando Maci\u00e1","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":["Article","BlogPosting"],"@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability#article","isPartOf":{"@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability"},"author":{"name":"Fernando Maci\u00e1","@id":"https:\/\/www.humanlevel.com\/en#\/schema\/person\/e85eb7565a8e3e5e682084fac681d8df"},"headline":"From blocking bots to negotiating access: the new approach to findability","datePublished":"2026-09-15T07:50:28+00:00","mainEntityOfPage":{"@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability"},"wordCount":1592,"commentCount":0,"publisher":{"@id":"https:\/\/www.humanlevel.com\/en#organization"},"image":{"@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability#primaryimage"},"thumbnailUrl":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-bots.jpg","articleSection":["SEO\/GEO"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability","url":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability","name":"From blocking bots to negotiating access: manage your findability | Human Level","isPartOf":{"@id":"https:\/\/www.humanlevel.com\/en#website"},"primaryImageOfPage":{"@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability#primaryimage"},"image":{"@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability#primaryimage"},"thumbnailUrl":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-bots.jpg","datePublished":"2026-09-15T07:50:28+00:00","description":"Learn how regulators, publishers, and CDNs are turning AI access to your content into negotiable value.","breadcrumb":{"@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability#primaryimage","url":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-bots.jpg","contentUrl":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/bloqueo-bots.jpg","width":272,"height":272,"caption":"From blocking bots to negotiating access"},{"@type":"BreadcrumbList","@id":"https:\/\/www.humanlevel.com\/en\/blog\/seo\/block-bots-negotiate-access-management-discoverability#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Inicio","item":"https:\/\/www.humanlevel.com\/en"},{"@type":"ListItem","position":2,"name":"Blog","item":"https:\/\/www.humanlevel.com\/en\/blog"},{"@type":"ListItem","position":3,"name":"SEO\/GEO","item":"https:\/\/www.humanlevel.com\/en\/blog\/seo"},{"@type":"ListItem","position":4,"name":"From blocking bots to negotiating access: the new approach to findability"}]},{"@type":"WebSite","@id":"https:\/\/www.humanlevel.com\/en#website","url":"https:\/\/www.humanlevel.com\/en","name":"Human Level","description":"Web positioning and online marketing consultant Human Level","publisher":{"@id":"https:\/\/www.humanlevel.com\/en#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.humanlevel.com\/en?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.humanlevel.com\/en#organization","name":"Human Level","url":"https:\/\/www.humanlevel.com\/en","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.humanlevel.com\/en#\/schema\/logo\/image\/","url":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/logohl25x3.png","contentUrl":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/logohl25x3.png","width":600,"height":93,"caption":"Human Level"},"image":{"@id":"https:\/\/www.humanlevel.com\/en#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/company\/human-level-communications","https:\/\/www.youtube.com\/user\/humanlevelcommunica","https:\/\/bsky.app\/profile\/humanlevel.bsky.social","https:\/\/instagram.com\/humanlevel"]},{"@type":"Person","@id":"https:\/\/www.humanlevel.com\/en#\/schema\/person\/e85eb7565a8e3e5e682084fac681d8df","name":"Fernando Maci\u00e1","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/1x1-fernando-macia-26-2-96x96.jpg","url":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/1x1-fernando-macia-26-2-96x96.jpg","contentUrl":"https:\/\/www.humanlevel.com\/wp-content\/uploads\/1x1-fernando-macia-26-2-96x96.jpg","caption":"Fernando Maci\u00e1"},"description":"Founder and CEO of Human Level. Expert SEO consultant with more than 20 years of experience. He has been a professor at numerous universities and business schools, and director of the Master in Professional SEO and SEM and the Advanced SEO Course at KSchool. Author of a dozen books on SEO and digital marketing.","sameAs":["https:\/\/www.amazon.es\/Fernando-Macia-Domene\/e\/B004G45KK6\/","https:\/\/www.linkedin.com\/in\/fernandomacia\/","https:\/\/x.com\/fernandomacia"],"url":"https:\/\/www.humanlevel.com\/en\/author\/fernando"}]}},"_links":{"self":[{"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/posts\/90942","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/comments?post=90942"}],"version-history":[{"count":4,"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/posts\/90942\/revisions"}],"predecessor-version":[{"id":90951,"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/posts\/90942\/revisions\/90951"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/media\/90941"}],"wp:attachment":[{"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/media?parent=90942"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/categories?post=90942"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.humanlevel.com\/en\/wp-json\/wp\/v2\/tags?post=90942"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}