remove attention abstraction (#5324) #234

Job	Run time
test-win-nightly (windows, 3.9, 12.1, nightly)	0s
test-win-nightly (windows, 3.10, 12.1, nightly)	0s
test-win-nightly (windows, 3.11, 12.1, nightly)	0s
test-win-nightly (windows, 3.12, 12.1, nightly)	0s
test-stable (macos, 3.9, 12.1, stable)	12m 58s
test-stable (macos, 3.10, 12.1, stable)	12m 32s
test-stable (macos, 3.11, 12.1, stable)	11m 58s
test-stable (macos, 3.12, 12.1, stable)	13m 19s
test-unix-nightly (macos, 3.11, 12.1, nightly)	13m 30s
test-stable (linux, 3.9, 12.1, stable)	6m 4s
test-unix-nightly (linux, 3.11, 12.1, nightly)	5m 58s
test-stable (linux, 3.10, 12.1, stable)	5m 54s
test-stable (linux, 3.11, 12.1, stable)	6m 16s
test-stable (linux, 3.12, 12.1, stable)	6m 4s
test-stable (windows, 3.9, 12.1, stable)	0s
test-stable (windows, 3.10, 12.1, stable)	0s
test-stable (windows, 3.11, 12.1, stable)	0s
test-stable (windows, 3.12, 12.1, stable)	0s
	1h 34m 33s

Provide feedback