[CI] Remove docker cache #206

Job	Run time
windows-latest - Python - sources - tests	3m 48s
macos-13 - Python - sources - tests	4m 17s
ubuntu-latest - Python - sources - tests	1m 28s
lint / Ubuntu - Python - lint	1m 24s
Source distribution - Python - deploy	0s
notify	0s
	10m 57s

Provide feedback