How to create an unbuffered pipe?

I’ve a script that reads various data and writes it in a pipe, so other scripts can look into it.

PIPE=~/observer.pipe
mkfifo $PIPE

exec 3<> $PIPE
~/observer > $PIPE &

This script is running all the time, and occasionally I want to see the data that comes as I am looking. So I check the content with

cat ~/observer.pipe

When I do that, a ton of stuff has buffered, but I am only interested in data as it comes in while watching the pipe. So no buffer, everything before can be lost.
I tried it with

stdbuf -o0 ~/observer > $PIPE &

But no change. Then I tried always watching

PIPE=~/observer.pipe
mkfifo $PIPE

exec 3<> $PIPE
~/observer > $PIPE &
cat < $PIPE > /dev/null

and read the pipe in my other scripts when I need it

cat < ~/observer.pipe

but that gets an error after the first data was read

cat: write error: Broken pipe

can there be only one listener?
What other way is available?

Update: This answer by @KamilMaciorowski gave me a very good workaround.