Re: infiniband limit of 32 cards per system?

[Roland Dreier <rdreier@cisco.com>]

 > My colleague points out the following enum in uverbs_main.c:
 > 
 > 	enum {
 > 		IB_UVERBS_MAJOR       = 231,
 > 		IB_UVERBS_BASE_MINOR  = 192,
 > 		IB_UVERBS_MAX_DEVICES = 32
 > 	};
 > 
 > Experimentally, we've determined that on a system where we
 > plugged in 40 IB cards, OFED only reports 32 cards are present.

wow, 40 HCAs in one system !

 > If that enum is indeed the limiting factor, would someone mind
 > explaining (or pointing me at TFM ;) why it's limited to 32
 > devices?

That dates back to when device #s had 8 bits for major and 8 bits for
minor.  We got one major assigned for IB, and had to split up the 256
minors that gave us among userspace verbs, management access, etc.  And
32 seemed like a pretty reasonable limit for most uses.

Nowadays I guess we should look into expanding that to dynamic device
numbers on overflow, assuming you do have a realistic situation where
someone would want to use that many adapters per system.

 - R.