Although there are repeated contents in two files, i.e., dependencies, we should understand that they serve completely different purposes. The same applies to requirements.txt and setup.py. See install_requires vs requirements files

pyptoject.toml describes the dependency through dependencies and optional-dependencies tables. The dependencies are used to build the project. It is designed for build tools. We can think of it as a listing of “Abstract” requirements that a project needs to run correctly.

However, one dependency may further depend on other dependencies. requirements.txt lists all pip install arguments in a file. It aims to show what packages are needed to configure the package running environment. In other words, requirements.txt tells you what packages are needed to achieve a complete environment. It often contains an exhaustive listing of pinned versions.

Note that requirements.txt is not used for building the package. We run pip install -r requirements.txt to configure the environment that supports the package running. But package itself is not built yet. It is more like a design document for deployment stuff, letting you know the environment.

If we simply write scripts and do not want to build a package, then requirements.txt is sufficient. We simply install necessary packages and we can run the script. But we need to pay attention that import and from ... import ... for self-written scripts are not good practice if we do not organize scripts in a package structure.