sys.modules is the Python-accessible reference to the canonical data structure for tracking what modules are imported. Removing a name from that dictionary means that any future attempts to import the module will result in Python loading the module from scratch.
In other words, whenever you use an import statement, the first thing Python does is check if the dictionary that sys.modules references already has an entry for that module and proceed with the next step (binding names in the current namespace) without loading the module first. If you delete entries from sys.modules, then Python won't find the already-loaded module and loads again.
Note the careful wording about Python-accessible here. The actual dictionary lives on the Python heap and sys.modules is just one reference to it. You replaced that reference with another dictionary by assigning to sys.modules. However, the interpreter has more references to it; they just are not accessible from Python code, not without ctypes trickery to access the C-API anyway.
Note that this is explicitly documented:
However, replacing the dictionary will not necessarily work as expected and deleting essential items from the dictionary may cause Python to fail.
From the C-API, you'd have to use PyThreadState_Get()->interp->modules to get the internal reference to the dictionary.