{"id":79901,"topic":"ai","source":"BBC","title":"Anthropic blocks 'malicious use' of AI that could develop biological weapons - BBC","url":"https://www.bbc.com/news/articles/cx2zrrpkx20o","url_hash":"13294a9e62582b96127f873a33531c3922d237ae","author":"","summary":"Anthropic blocks 'malicious use' of AI that could develop biological weapons BBC","content":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development.\nThe company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.\nThe cases of concern detected over the past eight months ranged from fake dating apps and hotel Wifi scams, to surveillance built to identify dissidents.\nAnthropic has also accused Chinese AI firms of trying to replicate Claude's capabilities.\nThe lengthy report lists cases in which suspected state-sponsored groups, criminals, spyware vendors, state propaganda institutions and politically motivated individuals misused its technology.\n\"Malicious use\" of its Claude Haiku, Sonnet, and Opus models was disrupted between December 2025 and August 2026, it said.\nNone of the misuse cases involved Claude Fable or the powerful Mythos-class models, with the exception of one instance of distillation - the process for training smaller AI models using larger, more expensive models.\nAs well as detecting the use of its models for cyber and influence operations, surveillance, scams and fraud and weapon development, Anthropic said it had blocked scientists who used its AI in ways that could support biological weapons development.\nThe report highlighted \"five case studies of actors using our models in ways that could support biological weapons development\".\nBiological misuse, it said, is \"one of the most serious risks of frontier AI model\". Without the correct safeguards, such capabilities \"could have catastrophic consequences\", Anthropic said.\n\"The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease,\" Anthropic said.\n\"You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody,'\" he said.\nThe report also noted six cases where Claude was used \"to develop software for conventional weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them\".\nThe report indicated that cybercriminals and state-backed hackers have increasingly used its technology to assist their operations.\nHacking group ShinyHunters, as well as China-based labs, were among those named in the report.\nThe report also said that a hacking group whose work is consistent with the Russia-based Midnight Blizzard allegedly used AI to build a system that automatically detected when its malware was flagged by security defences and rewrote code until it evaded detection.\nThe California-based company said it had incorporated its findings into its processes \"to better prevent, detect, and disrupt these activities in the future\".\nAnthropic said it had shared intelligence with authorities and industry partners where appropriate.\nThe report - the company's first this year - comes after a top safety researcher at Anthropic warned AI is advancing so quickly he believes there is a greater than 10% chance it \"could kill all humans\" within the next decade.\nIn an articled published on 6 September, OpenAI chief scientist Jakub Pachocki called for the industry to implement \"voluntary slowdowns\" until safeguards are set.\n\"I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,\" Pachocki said, adding that OpenAI, which makes ChatGPT, will continue its work on building safeguards, though broader interventions are required.\nThe warning prompted an open letter to UK Prime Minister Andy Burnham calling for a new multinational treaty for the safe development of AI and a call for governments to collaborate about what a treaty based on the development of superintelligence should look like.\nIn the US, Democratic lawmaker Bernie Sanders has introduced legislation to ban AI superintelligence and temporarily pause advanced AI development.\nWith additional reporting from Osmond Chia.","image_url":"https://ichef.bbci.co.uk/news/1024/branded_news/090c/live/88ca35f0-ad6d-11f1-8edd-ada2fcb84b27.jpg","lang":"en","published_at":"2026-09-11T02:46:30+00:00","fetched_at":"2026-09-11T03:15:04+00:00","status":"read","starred":0,"extract_state":"ok","summary_auto":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development. The company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.","cluster_id":null,"extract_retries":0,"extract_error":null,"contract_version":"news_item.v1","format_contract_version":"news_item_formats.v1","dedup_url":"https://www.bbc.com/news/articles/cx2zrrpkx20o","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4172 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4172,"summary_length":435,"usable_text_length":4172,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4172,"summary_length":435}},"news_item":{"id":79901,"canonical_url":"https://www.bbc.com/news/articles/cx2zrrpkx20o","source_url":"https://www.bbc.com/news/articles/cx2zrrpkx20o","title":"Anthropic blocks 'malicious use' of AI that could develop biological weapons - BBC","source_name":"BBC","author":null,"published_at":"2026-09-11T02:46:30+00:00","locale":"en","topic":"ai","tags":[],"rss_summary":"Anthropic blocks 'malicious use' of AI that could develop biological weapons BBC","full_text":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development.\nThe company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.\nThe cases of concern detected over the past eight months ranged from fake dating apps and hotel Wifi scams, to surveillance built to identify dissidents.\nAnthropic has also accused Chinese AI firms of trying to replicate Claude's capabilities.\nThe lengthy report lists cases in which suspected state-sponsored groups, criminals, spyware vendors, state propaganda institutions and politically motivated individuals misused its technology.\n\"Malicious use\" of its Claude Haiku, Sonnet, and Opus models was disrupted between December 2025 and August 2026, it said.\nNone of the misuse cases involved Claude Fable or the powerful Mythos-class models, with the exception of one instance of distillation - the process for training smaller AI models using larger, more expensive models.\nAs well as detecting the use of its models for cyber and influence operations, surveillance, scams and fraud and weapon development, Anthropic said it had blocked scientists who used its AI in ways that could support biological weapons development.\nThe report highlighted \"five case studies of actors using our models in ways that could support biological weapons development\".\nBiological misuse, it said, is \"one of the most serious risks of frontier AI model\". Without the correct safeguards, such capabilities \"could have catastrophic consequences\", Anthropic said.\n\"The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease,\" Anthropic said.\n\"You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody,'\" he said.\nThe report also noted six cases where Claude was used \"to develop software for conventional weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them\".\nThe report indicated that cybercriminals and state-backed hackers have increasingly used its technology to assist their operations.\nHacking group ShinyHunters, as well as China-based labs, were among those named in the report.\nThe report also said that a hacking group whose work is consistent with the Russia-based Midnight Blizzard allegedly used AI to build a system that automatically detected when its malware was flagged by security defences and rewrote code until it evaded detection.\nThe California-based company said it had incorporated its findings into its processes \"to better prevent, detect, and disrupt these activities in the future\".\nAnthropic said it had shared intelligence with authorities and industry partners where appropriate.\nThe report - the company's first this year - comes after a top safety researcher at Anthropic warned AI is advancing so quickly he believes there is a greater than 10% chance it \"could kill all humans\" within the next decade.\nIn an articled published on 6 September, OpenAI chief scientist Jakub Pachocki called for the industry to implement \"voluntary slowdowns\" until safeguards are set.\n\"I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,\" Pachocki said, adding that OpenAI, which makes ChatGPT, will continue its work on building safeguards, though broader interventions are required.\nThe warning prompted an open letter to UK Prime Minister Andy Burnham calling for a new multinational treaty for the safe development of AI and a call for governments to collaborate about what a treaty based on the development of superintelligence should look like.\nIn the US, Democratic lawmaker Bernie Sanders has introduced legislation to ban AI superintelligence and temporarily pause advanced AI development.\nWith additional reporting from Osmond Chia.","excerpt":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development. The company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.","extraction":{"state":"ok","confidence":0.9,"error":null,"explanation":"High confidence: full text extraction produced 4172 characters.","diagnostics_url":"/api/diagnose?url=https%3A//www.bbc.com/news/articles/cx2zrrpkx20o","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4172 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4172,"summary_length":435,"usable_text_length":4172,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4172,"summary_length":435}}},"display_formats":["compact","card","full","digest_section","json"]},"daily_stack_record":{"title":"Anthropic blocks 'malicious use' of AI that could develop biological weapons - BBC","url":"https://www.bbc.com/news/articles/cx2zrrpkx20o","summary":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development. The company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.","source":"BBC","date":"2026-09-11T02:46:30+00:00","content":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development.\nThe company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.\nThe cases of concern detected over the past eight months ranged from fake dating apps and hotel Wifi scams, to surveillance built to identify dissidents.\nAnthropic has also accused Chinese AI firms of trying to replicate Claude's capabilities.\nThe lengthy report lists cases in which suspected state-sponsored groups, criminals, spyware vendors, state propaganda institutions and politically motivated individuals misused its technology.\n\"Malicious use\" of its Claude Haiku, Sonnet, and Opus models was disrupted between December 2025 and August 2026, it said.\nNone of the misuse cases involved Claude Fable or the powerful Mythos-class models, with the exception of one instance of distillation - the process for training smaller AI models using larger, more expensive models.\nAs well as detecting the use of its models for cyber and influence operations, surveillance, scams and fraud and weapon development, Anthropic said it had blocked scientists who used its AI in ways that could support biological weapons development.\nThe report highlighted \"five case studies of actors using our models in ways that could support biological weapons development\".\nBiological misuse, it said, is \"one of the most serious risks of frontier AI model\". Without the correct safeguards, such capabilities \"could have catastrophic consequences\", Anthropic said.\n\"The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease,\" Anthropic said.\n\"You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody,'\" he said.\nThe report also noted six cases where Claude was used \"to develop software for conventional weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them\".\nThe report indicated that cybercriminals and state-backed hackers have increasingly used its technology to assist their operations.\nHacking group ShinyHunters, as well as China-based labs, were among those named in the report.\nThe report also said that a hacking group whose work is consistent with the Russia-based Midnight Blizzard allegedly used AI to build a system that automatically detected when its malware was flagged by security defences and rewrote code until it evaded detection.\nThe California-based company said it had incorporated its findings into its processes \"to better prevent, detect, and disrupt these activities in the future\".\nAnthropic said it had shared intelligence with authorities and industry partners where appropriate.\nThe report - the company's first this year - comes after a top safety researcher at Anthropic warned AI is advancing so quickly he believes there is a greater than 10% chance it \"could kill all humans\" within the next decade.\nIn an articled published on 6 September, OpenAI chief scientist Jakub Pachocki called for the industry to implement \"voluntary slowdowns\" until safeguards are set.\n\"I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,\" Pachocki said, adding that OpenAI, which makes ChatGPT, will continue its work on building safeguards, though broader interventions are required.\nThe warning prompted an open letter to UK Prime Minister Andy Burnham calling for a new multinational treaty for the safe development of AI and a call for governments to collaborate about what a treaty based on the development of superintelligence should look like.\nIn the US, Democratic lawmaker Bernie Sanders has introduced legislation to ban AI superintelligence and temporarily pause advanced AI development.\nWith additional reporting from Osmond Chia.","confidence":0.9,"diagnostics_url":"/api/diagnose?url=https%3A//www.bbc.com/news/articles/cx2zrrpkx20o","quality_bucket":"high","failure_kind":"none","retryable":false,"quality_reason":"High confidence: full text extraction produced 4172 characters.","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4172 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4172,"summary_length":435,"usable_text_length":4172,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4172,"summary_length":435}},"tags":[]},"fallback_formats":["markdown","json","html"],"actions":{"read":"/item/79901","export_markdown":"/api/items/79901/export?format=markdown","export_json":"/api/items/79901/export?format=json","diagnose":"/api/diagnose?url=https%3A//www.bbc.com/news/articles/cx2zrrpkx20o"},"formats":{"full":{"id":79901,"title":"Anthropic blocks 'malicious use' of AI that could develop biological weapons - BBC","url":"https://www.bbc.com/news/articles/cx2zrrpkx20o","source":"BBC","author":null,"published_at":"2026-09-11T02:46:30+00:00","locale":"en","topic":"ai","tags":[],"excerpt":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development. The company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.","full_text":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development.\nThe company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.\nThe cases of concern detected over the past eight months ranged from fake dating apps and hotel Wifi scams, to surveillance built to identify dissidents.\nAnthropic has also accused Chinese AI firms of trying to replicate Claude's capabilities.\nThe lengthy report lists cases in which suspected state-sponsored groups, criminals, spyware vendors, state propaganda institutions and politically motivated individuals misused its technology.\n\"Malicious use\" of its Claude Haiku, Sonnet, and Opus models was disrupted between December 2025 and August 2026, it said.\nNone of the misuse cases involved Claude Fable or the powerful Mythos-class models, with the exception of one instance of distillation - the process for training smaller AI models using larger, more expensive models.\nAs well as detecting the use of its models for cyber and influence operations, surveillance, scams and fraud and weapon development, Anthropic said it had blocked scientists who used its AI in ways that could support biological weapons development.\nThe report highlighted \"five case studies of actors using our models in ways that could support biological weapons development\".\nBiological misuse, it said, is \"one of the most serious risks of frontier AI model\". Without the correct safeguards, such capabilities \"could have catastrophic consequences\", Anthropic said.\n\"The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease,\" Anthropic said.\n\"You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody,'\" he said.\nThe report also noted six cases where Claude was used \"to develop software for conventional weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them\".\nThe report indicated that cybercriminals and state-backed hackers have increasingly used its technology to assist their operations.\nHacking group ShinyHunters, as well as China-based labs, were among those named in the report.\nThe report also said that a hacking group whose work is consistent with the Russia-based Midnight Blizzard allegedly used AI to build a system that automatically detected when its malware was flagged by security defences and rewrote code until it evaded detection.\nThe California-based company said it had incorporated its findings into its processes \"to better prevent, detect, and disrupt these activities in the future\".\nAnthropic said it had shared intelligence with authorities and industry partners where appropriate.\nThe report - the company's first this year - comes after a top safety researcher at Anthropic warned AI is advancing so quickly he believes there is a greater than 10% chance it \"could kill all humans\" within the next decade.\nIn an articled published on 6 September, OpenAI chief scientist Jakub Pachocki called for the industry to implement \"voluntary slowdowns\" until safeguards are set.\n\"I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,\" Pachocki said, adding that OpenAI, which makes ChatGPT, will continue its work on building safeguards, though broader interventions are required.\nThe warning prompted an open letter to UK Prime Minister Andy Burnham calling for a new multinational treaty for the safe development of AI and a call for governments to collaborate about what a treaty based on the development of superintelligence should look like.\nIn the US, Democratic lawmaker Bernie Sanders has introduced legislation to ban AI superintelligence and temporarily pause advanced AI development.\nWith additional reporting from Osmond Chia.","reading_time_min":3,"extraction":{"state":"ok","confidence":0.9,"error":null,"explanation":"High confidence: full text extraction produced 4172 characters.","diagnostics_url":"/api/diagnose?url=https%3A//www.bbc.com/news/articles/cx2zrrpkx20o","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4172 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4172,"summary_length":435,"usable_text_length":4172,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4172,"summary_length":435}}},"quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4172 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4172,"summary_length":435,"usable_text_length":4172,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4172,"summary_length":435}},"actions":{"read":"/item/79901","export_markdown":"/api/items/79901/export?format=markdown","export_json":"/api/items/79901/export?format=json","diagnose":"/api/diagnose?url=https%3A//www.bbc.com/news/articles/cx2zrrpkx20o"}},"digest":{"id":79901,"title":"Anthropic blocks 'malicious use' of AI that could develop biological weapons - BBC","url":"https://www.bbc.com/news/articles/cx2zrrpkx20o","source":"BBC","topic":"ai","published_at":"2026-09-11T02:46:30+00:00","excerpt":"Anthropic blocks 'malicious use' of AI that could develop biological weapons Anthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development. The company's AI model Claude was…","quality_bucket":"high","quality_reason":"High confidence: full text extraction produced 4172 characters.","reading_time_min":3,"cluster_id":null},"card":{"display_title":"Anthropic blocks 'malicious use' of AI that could develop biological weapons - BBC","subtitle":"BBC · 2026-09-11","summary":"Anthropic blocks 'malicious use' of AI that could develop biological weapons Anthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and…","badges":["quality:high"],"links":{"read":"/item/79901","original":"https://www.bbc.com/news/articles/cx2zrrpkx20o","diagnose":"/api/diagnose?url=https%3A//www.bbc.com/news/articles/cx2zrrpkx20o"},"quality_warning":null},"export":{"title":"Anthropic blocks 'malicious use' of AI that could develop biological weapons - BBC","url":"https://www.bbc.com/news/articles/cx2zrrpkx20o","summary":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development. The company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.","source":"BBC","date":"2026-09-11T02:46:30+00:00","content":"Anthropic blocks 'malicious use' of AI that could develop biological weapons\nAnthropic says it has identified and disrupted attempts to use its AI model for \"malicious activity\" that could support biological and conventional weapons development.\nThe company's AI model Claude was also used in a Russia-linked cyber espionage campaign and by an Iranian propaganda institution, according to Anthropic's latest threat intelligence report.\nThe cases of concern detected over the past eight months ranged from fake dating apps and hotel Wifi scams, to surveillance built to identify dissidents.\nAnthropic has also accused Chinese AI firms of trying to replicate Claude's capabilities.\nThe lengthy report lists cases in which suspected state-sponsored groups, criminals, spyware vendors, state propaganda institutions and politically motivated individuals misused its technology.\n\"Malicious use\" of its Claude Haiku, Sonnet, and Opus models was disrupted between December 2025 and August 2026, it said.\nNone of the misuse cases involved Claude Fable or the powerful Mythos-class models, with the exception of one instance of distillation - the process for training smaller AI models using larger, more expensive models.\nAs well as detecting the use of its models for cyber and influence operations, surveillance, scams and fraud and weapon development, Anthropic said it had blocked scientists who used its AI in ways that could support biological weapons development.\nThe report highlighted \"five case studies of actors using our models in ways that could support biological weapons development\".\nBiological misuse, it said, is \"one of the most serious risks of frontier AI model\". Without the correct safeguards, such capabilities \"could have catastrophic consequences\", Anthropic said.\n\"The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease,\" Anthropic said.\n\"You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody,'\" he said.\nThe report also noted six cases where Claude was used \"to develop software for conventional weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them\".\nThe report indicated that cybercriminals and state-backed hackers have increasingly used its technology to assist their operations.\nHacking group ShinyHunters, as well as China-based labs, were among those named in the report.\nThe report also said that a hacking group whose work is consistent with the Russia-based Midnight Blizzard allegedly used AI to build a system that automatically detected when its malware was flagged by security defences and rewrote code until it evaded detection.\nThe California-based company said it had incorporated its findings into its processes \"to better prevent, detect, and disrupt these activities in the future\".\nAnthropic said it had shared intelligence with authorities and industry partners where appropriate.\nThe report - the company's first this year - comes after a top safety researcher at Anthropic warned AI is advancing so quickly he believes there is a greater than 10% chance it \"could kill all humans\" within the next decade.\nIn an articled published on 6 September, OpenAI chief scientist Jakub Pachocki called for the industry to implement \"voluntary slowdowns\" until safeguards are set.\n\"I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,\" Pachocki said, adding that OpenAI, which makes ChatGPT, will continue its work on building safeguards, though broader interventions are required.\nThe warning prompted an open letter to UK Prime Minister Andy Burnham calling for a new multinational treaty for the safe development of AI and a call for governments to collaborate about what a treaty based on the development of superintelligence should look like.\nIn the US, Democratic lawmaker Bernie Sanders has introduced legislation to ban AI superintelligence and temporarily pause advanced AI development.\nWith additional reporting from Osmond Chia.","confidence":0.9,"diagnostics_url":"/api/diagnose?url=https%3A//www.bbc.com/news/articles/cx2zrrpkx20o","quality_bucket":"high","failure_kind":"none","retryable":false,"quality_reason":"High confidence: full text extraction produced 4172 characters.","quality_profile":{"profile_version":"extraction_quality.v2","bucket":"high","confidence":0.9,"failure_kind":"none","retryable":false,"retry_after_attempts":0,"reason":"High confidence: full text extraction produced 4172 characters.","operator_guidance":{"severity":"ok","recommended_action":"trust_full_text","next_step":"Use the extracted full text as the primary article source.","operator_label":"Ready","can_retry":false,"can_use_summary":false,"diagnostics_required":false},"content_depth":{"contract_version":"content_depth.v1","category":"full_text","label":"Full text","has_full_text":true,"has_summary":true,"content_length":4172,"summary_length":435,"usable_text_length":4172,"source_field":"content"},"legacy_collapsed":false,"signals":{"extract_state":"ok","extract_error":null,"extract_retries":0,"content_length":4172,"summary_length":435}},"tags":[],"format_contract_version":"news_item_formats.v1"}}}