TypeError: Descriptors cannot not be created directly [Fixed]

# TypeError: Descriptors cannot not be created directly [Fixed]

The error "TypeError: Descriptors cannot not be created directly" occurs because there has been a breaking change in protobuf version 4.

To solve the error, pin your protobuf version to 3.20.*.

Copied!
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

The error occurs because some breaking changes have been introduced with protobuf version 4.21.0.

The version is a new major update following 3.20.1 and is based on the upb library to offer better parsing performance.

You can solve the error by doing one of three things:

1. Install the latest 3.20.X release of protobuf and update your requirements.txt file to pin the version (recommended).
2. Set the PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION environment variable.
3. Upgrade the version of the packages that caused the issue in the hope that a fix has been implemented.

# Install the latest 3.20 release of protobuf

Open your terminal and run the following command to pin your protobuf version to the last 3.20 release.

Copied!
pip uninstall protobuf
pip install "protobuf==3.20.*"

pip3 uninstall protobuf
pip3 install "protobuf==3.20.*"

python -m pip uninstall protobuf
python -m pip install "protobuf==3.20.*"

python3 -m pip uninstall protobuf
python3 -m pip install "protobuf==3.20.*"

py -m pip uninstall protobuf
py -m pip install "protobuf==3.20.*"

When you run the command, you might get an error that states "ERROR: pip's dependency resolver does not currently take into account all the packages that are installed.".

Once you run the installation command, add the following line to your requirements.txt file.

Copied!
protobuf==3.20.*

You can run the pip show command to make sure your protobuf version is pinned to 3.20.X.

Copied!
pip show protobuf

pip3 show protobuf

If the version of protobuf hasn't been set to 3.20.*, run the command with the --force-reinstall option.

Copied!
pip install "protobuf==3.20.*" --force-reinstall

pip3 install "protobuf==3.20.*" --force-reinstall

# Upgrade the version of the packages that cause the issue

If none of the suggestions helped, try to upgrade the package that causes the issue.

Make sure to replace tensorflow with the name of the package that caused the issue in your case.

Copied!
pip install tensorflow --upgrade
pip3 install tensorflow --upgrade

python -m pip install tensorflow --upgrade
python3 -m pip install tensorflow --upgrade
py -m pip install tensorflow --upgrade

As the error states:

If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.

# Setting the PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION environment variable

An alternative way to solve the error is to set the PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION environment variable to python as shown in the error message.

Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

Copied!
# Linux and MacOS
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python

# Windows Command Prompt
setx PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION python

# PowerShell
$Env:PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION="python"

The newer version of protobuf uses the upb library under the hood and offers significantly better parsing performance than previous releases for large payloads.

However, we don't have much of a choice as of now.

The official suggestion is that version 3.20.X should be used for now.

A feature request for adding a backward compatibility layer has been raised.

# A note about Google App Engine

You might get the error in Google App Engine even if you don't use the protobuf package directly.

Adding the following line to your requirements.txt file should fix the error.

Copied!
protobuf==3.20.*

The reason the error occurs is that the protobuf package has released a new major version with some breaking changes.

Google didn't regenerate its libraries with the latest protobuf version which caused the error.

You have to pin your protobuf version to 3.20.* until Google makes the necessary changes.

As the error message states:

If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.

If you cannot immediately regenerate your protos, the possible workarounds are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python.

If none of the suggestions helped, you can try to upgrade all packages in your environment.

# Upgrade all packages in your environment

Upgrading all packages in your environment might help because if the error is caused in an older version of a package that uses protobuf, a fix might have been implemented in more recent versions.

The most straightforward way to upgrade all outdated packages is to use a Python script.

Copied!
import pkg_resources
from subprocess import call

packages = [dist.project_name for dist in pkg_resources.working_set]
call("pip install --upgrade " + ' '.join(packages), shell=True)

Here are alternative commands you can use to upgrade all outdated packages.

Copied!
# 👇️ macOS or Linux
pip install -U `pip list --outdated | awk 'NR>2 {print $1}'`

# 👇️ Windows
for /F "delims= " %i in ('pip list --outdated') do pip install -U %i

If you use a requirements.txt file, you can update it with the following command.

Copied!
pip freeze > requirements.txt

There is more information about why the error occurs in this GitHub issue.

The suggestion is that version 3.20.X should be used for now and a feature request for adding a backward compatibility layer has been raised.