There are some genetics to it, no doubt. They also were obviously "used" at one point to be workhorses. (no racist, just fact) But, some sports fit their natural skill sets perfectly. It's just the way it is. Being slaves certainly didn't hurt as they were worked hard daily for years.
That said, I don't think after this many years the actual labor make them better at sports. I think it's more genetics than breeding.
But, then again, it's one of those things that can't be proven, so it's another cyclical argument.