We need to regulate or control that.
In more technical terms, an AI agent has a goal set. What if the goal is the destruction of humanity, or a subset thereof?
For an example of AI gone bad, see
https://www.politico.com/news/2026/08/04/anthropic-openai-aisi-testing-01025042
-------
Regarding
https://www.politico.com/news/2026/09/16/pentagon-ai-congress-worries-defense-01080418 :
How can we be 100% sure that any AI in nuclear command-and-control systems won't start a nuclear war? Where is the fail-safe assurance?
Wikipedia has a very extensive discussion of these issues here:
https://en.wikipedia.org/wiki/Existential_risk_from_artificial_intelligence?wprov=sfla1