Spaces:
Running on Zero
Running on Zero
Add Black Forest Labs content moderation
#64
by alexanderwangus - opened
Why
This demo generates images from any prompt and uploaded image without a content filter. This PR adds Black Forest Labs' moderation to every generation.
What changes
- At startup, the Space downloads the shared client
bfl_moderation.pyfrom black-forest-labs/hf-spaces-moderation-client. - Inside the
@spaces.GPUfunction, the prompt and any input images are checked while the image generates, and the finished image is checked before it is returned. A blocked request shows "This request was blocked by our content filter." and returns no image. - If the input check cannot run, the request continues, since the output check still runs. If the output check cannot run, the request fails with "The content filter is unavailable, please try again later."
- The checks run inside the GPU function, so they count against the caller's ZeroGPU quota:
inferandinfer_examplegetduration=70. - Pins
sentencepiece==0.2.2. The Space builder's base image pinsprotobuf<4, and sentencepiece 0.2.3 importsgoogle.protobuf.runtime_version, which needs a newer protobuf, so any rebuild crashed at startup withImportError: cannot import name 'runtime_version' from 'google.protobuf'. - Pins
diffusers==0.41.0instead of installing from gitmain. diffusersmain(and 0.42.1 onward) requires Python 3.11, and this Space builds on Python 3.10, so a rebuild without a cached install failed withPackage 'diffusers' requires a different Python: 3.10.13 not in '>=3.11.0'. 0.41.0 is the newest release for Python 3.10 and includes this Space's pipeline.
The API key comes from the BFL_MODERATION_KEY Space secret, which is already set on this Space.
Testing
On a private copy of this Space: benign text-to-image and benign edit → image (18.0 s, 32.0 s); explicit prompt and gory edit → blocked.
alexanderwangus changed pull request status to merged