Re: linux rdma 3.14 merge plans

Re: linux rdma 3.14 merge plans

[Or Gerlitz]

On Mon, Jan 20, 2014, Or Gerlitz <or.gerlitz@gmail.com> wrote:
> On Sun, Jan 19, 2014, sagi grimberg <sagig@mellanox.com> wrote:
>> Thanks Nic,  let me elaborate on this,
[...]
>> Hope this helps,

> Hi Roland, with Nic's && Sagi's answers @ hand, were your questions resolved?

Roland, ping! the signature patches were posted > three months ago. We
deserve a response from the maintainer that goes beyond "I need to
think on that".

Responsiveness was stated by Linus to be the #1 requirement from
kernel maintainers.

Or.

Re: linux rdma 3.14 merge plans

[Roland Dreier]

On Tue, Jan 21, 2014 at 2:00 PM, Or Gerlitz <or.gerlitz@gmail.com> wrote:
> Roland, ping! the signature patches were posted > three months ago. We
> deserve a response from the maintainer that goes beyond "I need to
> think on that".
>
> Responsiveness was stated by Linus to be the #1 requirement from
> kernel maintainers.

Or, I'm not sure what response you're after from me.  Linus has also
said that maintainers should say "no" a lot more
(http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I
won't merge this patch set, since it adds a bunch of complexity to
support a feature no one really cares about."  Is that it?  (And yes I
am skeptical about this stuff — I work at an enterprise storage
company and even here it's hard to find anyone who cares about
DIF/DIX, especially offload features that stop it from being
end-to-end)

I'm sure you're not expecting me to say, "Sure, I'll merge it without
understanding the problem it's solving or how it's doing that,"
especially given the your recent history of pushing me to merge stuff
like the IP-RoCE patches back when they broke the userspace ABI.

I'd really rather spend my time on something actually useful like
cleaning up softroce.

 - R.

Re: linux rdma 3.14 merge plans

[Or Gerlitz]

On Wed, Jan 22, 2014 at 2:43 AM, Roland Dreier <roland@kernel.org> wrote:
> On Tue, Jan 21, 2014 at 2:00 PM, Or Gerlitz <or.gerlitz@gmail.com> wrote:
>> Roland, ping! the signature patches were posted > three months ago. We
>> deserve a response from the maintainer that goes beyond "I need to
>> think on that".
>>
>> Responsiveness was stated by Linus to be the #1 requirement from
>> kernel maintainers.
>
> Or, I'm not sure what response you're after from me.

Roland, what I am after is a r-e-s-p-o-n-s-e from you, and let it
contain what ever justified and/or unjustified mud as below. We posted
the V0 series on Oct 15 2013 and since that time not a word from you,
except for an "I need to think on that" comment last week after we
nudged million times.

You can't leave us clueless in the air for whole three months without
any concrete or unconcrete comment. There's no way to carry kernel
development like that. I am old enough to hear and face "no" and "wTF
is this" or "yTF you do it this way" etc etc, this happened few times
with e.g with networking patches we sent  and we either improved
things or did them differently or whatever needed to be done.

There's no way on earth to face plain ignoring of your work, and this
is what happens here. I had no way to get your below response except
for going to LKML, why?


> Linus has also said that maintainers should say "no" a lot more
> (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I
> won't merge this patch set, since it adds a bunch of complexity to
> support a feature no one really cares about."  Is that it?  (And yes I
> am skeptical about this stuff — I work at an enterprise storage
> company and even here it's hard to find anyone who cares about
> DIF/DIX, especially offload features that stop it from being
> end-to-end)
>
> I'm sure you're not expecting me to say, "Sure, I'll merge it without
> understanding the problem it's solving or how it's doing that,"
> especially given the your recent history of pushing me to merge stuff
> like the IP-RoCE patches back when they broke the userspace ABI.
>
> I'd really rather spend my time on something actually useful like
> cleaning up softroce.
>
>  - R.

Re: linux rdma 3.14 merge plans

[Nicholas A. Bellinger]

Roland & Co,

On Tue, 2014-01-21 at 16:43 -0800, Roland Dreier wrote:
> On Tue, Jan 21, 2014 at 2:00 PM, Or Gerlitz <or.gerlitz@gmail.com> wrote:
> > Roland, ping! the signature patches were posted > three months ago. We
> > deserve a response from the maintainer that goes beyond "I need to
> > think on that".
> >
> > Responsiveness was stated by Linus to be the #1 requirement from
> > kernel maintainers.
> 
> Or, I'm not sure what response you're after from me.  Linus has also
> said that maintainers should say "no" a lot more
> (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I
> won't merge this patch set, since it adds a bunch of complexity to
> support a feature no one really cares about."  Is that it? 

The patch set proposed by Sagi + Or is modest in terms of LOC to core IB
code, and includes mostly mlx5 specific driver changes that enables HW
offloads.

> (And yes I
> am skeptical about this stuff — I work at an enterprise storage
> company and even here it's hard to find anyone who cares about
> DIF/DIX, especially offload features that stop it from being
> end-to-end)
> 

My understanding is most HBAs capable of T10 PI offload in DIX PASS +
VERIFY mode are already implementing DIX INSERT + STRIP modes in various
capacities to support legacy environments.

Beyond the DIX INSERT + STRIP case for enterprise storage, the amount of
FC + SAS HBAs that already support T10 PI metadata is substantial.

> I'm sure you're not expecting me to say, "Sure, I'll merge it without
> understanding the problem it's solving or how it's doing that,"
> especially given the your recent history of pushing me to merge stuff
> like the IP-RoCE patches back when they broke the userspace ABI.

With the merge window now upon us, there is a understandable reluctance
to merge new features.  Given the amount of time the series has spent on
the list, it is however a good candidate to consider for an exception.

Short of that, are you planning to accept the series for the next round
once the current merge window closes..?

We'd really like to start enabling fabrics with these types of offloads
for v3.15. 

> I'd really rather spend my time on something actually useful like
> cleaning up softroce.
> 

+1 for softroce + T10 PI support!

--nab

Re: linux rdma 3.14 merge plans

[Nicholas A. Bellinger]

Hi Roland,

On Tue, 2014-01-21 at 23:27 -0800, Nicholas A. Bellinger wrote:
> Roland & Co,
> 
> On Tue, 2014-01-21 at 16:43 -0800, Roland Dreier wrote:
> > On Tue, Jan 21, 2014 at 2:00 PM, Or Gerlitz <or.gerlitz@gmail.com> wrote:
> > > Roland, ping! the signature patches were posted > three months ago. We
> > > deserve a response from the maintainer that goes beyond "I need to
> > > think on that".
> > >
> > > Responsiveness was stated by Linus to be the #1 requirement from
> > > kernel maintainers.
> > 
> > Or, I'm not sure what response you're after from me.  Linus has also
> > said that maintainers should say "no" a lot more
> > (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I
> > won't merge this patch set, since it adds a bunch of complexity to
> > support a feature no one really cares about."  Is that it? 
> 
> The patch set proposed by Sagi + Or is modest in terms of LOC to core IB
> code, and includes mostly mlx5 specific driver changes that enables HW
> offloads.
> 
> > (And yes I
> > am skeptical about this stuff — I work at an enterprise storage
> > company and even here it's hard to find anyone who cares about
> > DIF/DIX, especially offload features that stop it from being
> > end-to-end)
> > 
> 
> My understanding is most HBAs capable of T10 PI offload in DIX PASS +
> VERIFY mode are already implementing DIX INSERT + STRIP modes in various
> capacities to support legacy environments.
> 
> Beyond the DIX INSERT + STRIP case for enterprise storage, the amount of
> FC + SAS HBAs that already support T10 PI metadata is substantial.
> 
> > I'm sure you're not expecting me to say, "Sure, I'll merge it without
> > understanding the problem it's solving or how it's doing that,"
> > especially given the your recent history of pushing me to merge stuff
> > like the IP-RoCE patches back when they broke the userspace ABI.
> 
> With the merge window now upon us, there is a understandable reluctance
> to merge new features.  Given the amount of time the series has spent on
> the list, it is however a good candidate to consider for an exception.
> 
> Short of that, are you planning to accept the series for the next round
> once the current merge window closes..?
> 
> We'd really like to start enabling fabrics with these types of offloads
> for v3.15. 
> 

Now with the initial DIF backend taraget support in place for v3.14-rc1
code, we'd like to move forward on iser-target related pieces for T10
PI.

Can you give us an estimate of when you'll have some time to give
feedback on the outstanding patches..?

--nab

Re: linux rdma 3.14 merge plans

[Roland Dreier]

On Thu, Feb 6, 2014 at 4:02 PM, Nicholas A. Bellinger
<nab@linux-iscsi.org> wrote:
> Can you give us an estimate of when you'll have some time to give
> feedback on the outstanding patches..?

I hope to get to it in the next few weeks.

 - R.

Re: linux rdma 3.14 merge plans

[Or Gerlitz]

On Fri, Feb 7, 2014 at 2:04 AM, Roland Dreier <roland@kernel.org> wrote:
> On Thu, Feb 6, 2014 at 4:02 PM, Nicholas A. Bellinger
> <nab@linux-iscsi.org> wrote:
>> Can you give us an estimate of when you'll have some time to give
>> feedback on the outstanding patches..?
>
> I hope to get to it in the next few weeks.

Hi Roland,

So days, weeks and months are passing by and we still didn't get any
real feedback from you on the patches which now make a complete story:
verbs, driver, target and initiator nor on the detailed response/s
sent to you as replies to the few on the surface quick questions and
doubts you raised. 3.15 is coming soon and there's no reason to miss
it just for that lack of feedback as happened for 3.13 and 3.14 -- are
you picking on this or maybe prefer that Nic will push it upstream
through his tree? all the patches are now on the rdma-dif of the
target-pending tree, please let us know.




>
>  - R.

Re: linux rdma 3.14 merge plans

[Nicholas A. Bellinger]

On Thu, 2014-02-06 at 16:04 -0800, Roland Dreier wrote:
> On Thu, Feb 6, 2014 at 4:02 PM, Nicholas A. Bellinger
> <nab@linux-iscsi.org> wrote:
> > Can you give us an estimate of when you'll have some time to give
> > feedback on the outstanding patches..?
> 
> I hope to get to it in the next few weeks.
> 
>  - R.

Hi Roland,

We'd very much like to move forward with the verbs + mlx5 + LIO target
PI related patches for v3.15 code.

Given the verbs + mlx5 changes have undergone ~5 months of review at
this point, I'd like to go ahead and get them in target-pending/for-next
ASAP to continue making forward progress towards a mainline merge.

I'm happy with the state of the iser-target related changes, and Sagi &
Co are making good progress on the iser-initiator related patches with
Mike.

That all said, do you have an objection wrt taking this bits through
target-pending..?  Given the dependencies involved, that would seem the
most logical path to take.

--nab 

Re: linux rdma 3.14 merge plans

[Roland Dreier]

On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger
<nab@linux-iscsi.org> wrote:
> That all said, do you have an objection wrt taking this bits through
> target-pending..?  Given the dependencies involved, that would seem the
> most logical path to take.

Perhaps not surprisingly, I would prefer to get a chance to review a
major change to the core RDMA midlayer rather than having you merge it
through your tree.  So yes I do object.  Please give me a chance to
review and merge it.  I am working on that this week.

 - R.

Re: linux rdma 3.14 merge plans

[Or Gerlitz]

On 05/03/2014 17:18, Roland Dreier wrote:
> On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger
> <nab@linux-iscsi.org>  wrote:
>> >That all said, do you have an objection wrt taking this bits through
>> >target-pending..?  Given the dependencies involved, that would seem the
>> >most logical path to take.
> Perhaps not surprisingly, I would prefer to get a chance to review a
> major change to the core RDMA midlayer rather than having you merge it
> through your tree.  So yes I do object.  Please give me a chance to
> review and merge it.  I am working on that this week.

Hi Roland, we're very happy to hear that!!

As you know, we are soon marking whole five months!! (patches posted Oct 
15th/2013) of sitting and waiting to your feedback, lost few upstream 
releases on the way. Let's get this into the works.

Or.

Or.

Re: linux rdma 3.14 merge plans

[Nicholas A. Bellinger]

On Wed, 2014-03-05 at 07:18 -0800, Roland Dreier wrote:
> On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger
> <nab@linux-iscsi.org> wrote:
> > That all said, do you have an objection wrt taking this bits through
> > target-pending..?  Given the dependencies involved, that would seem the
> > most logical path to take.
> 
> Perhaps not surprisingly, I would prefer to get a chance to review a
> major change to the core RDMA midlayer rather than having you merge it
> through your tree.  So yes I do object.  Please give me a chance to
> review and merge it.  I am working on that this week.
> 

Great.  We'll be looking for a response by the end of the week.

Otherwise if you end up not having time, we'd still like to move forward
for v3.15 given the amount of review the series has already gotten on
the list.

Thank you,

--nab

RE: linux rdma 3.14 merge plans

[Devesh Sharma]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1744 bytes --]

Hi Roland,

Is it okay to send next series of patches even if previous series is not accepted yet in your tree? Off-course I will cut patches on top of previous series of patches.

-Regards
 Devesh

-----Original Message-----
From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma-owner@vger.kernel.org] On Behalf Of Nicholas A. Bellinger
Sent: Thursday, March 06, 2014 12:34 AM
To: Roland Dreier
Cc: Or Gerlitz; Hefty Sean; linux-rdma; Martin K. Petersen; target-devel; Sagi Grimberg; linux-kernel
Subject: Re: linux rdma 3.14 merge plans

On Wed, 2014-03-05 at 07:18 -0800, Roland Dreier wrote:
> On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger 
> <nab@linux-iscsi.org> wrote:
> > That all said, do you have an objection wrt taking this bits through 
> > target-pending..?  Given the dependencies involved, that would seem 
> > the most logical path to take.
> 
> Perhaps not surprisingly, I would prefer to get a chance to review a 
> major change to the core RDMA midlayer rather than having you merge it 
> through your tree.  So yes I do object.  Please give me a chance to 
> review and merge it.  I am working on that this week.
> 

Great.  We'll be looking for a response by the end of the week.

Otherwise if you end up not having time, we'd still like to move forward for v3.15 given the amount of review the series has already gotten on the list.

Thank you,

--nab



--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: linux rdma 3.14 merge plans

[Roland Dreier]

Sure, no problem.

Do you have a git tree with the latest versions of all the changes you
want for 3.15 in a branch?  That would be helpful as I catch up on
applying things, so that I don't miss anything.

If you don't have one, taking a little time to set one up on github or
wherever would be nice.  You can base your set of changes on Linus's
latest tree.

Thanks!
  Roland

On Thu, Mar 6, 2014 at 9:07 PM, Devesh Sharma <Devesh.Sharma@emulex.com> wrote:
> Hi Roland,
>
> Is it okay to send next series of patches even if previous series is not accepted yet in your tree? Off-course I will cut patches on top of previous series of patches.
>
> -Regards
>  Devesh
>
> -----Original Message-----
> From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma-owner@vger.kernel.org] On Behalf Of Nicholas A. Bellinger
> Sent: Thursday, March 06, 2014 12:34 AM
> To: Roland Dreier
> Cc: Or Gerlitz; Hefty Sean; linux-rdma; Martin K. Petersen; target-devel; Sagi Grimberg; linux-kernel
> Subject: Re: linux rdma 3.14 merge plans
>
> On Wed, 2014-03-05 at 07:18 -0800, Roland Dreier wrote:
>> On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger
>> <nab@linux-iscsi.org> wrote:
>> > That all said, do you have an objection wrt taking this bits through
>> > target-pending..?  Given the dependencies involved, that would seem
>> > the most logical path to take.
>>
>> Perhaps not surprisingly, I would prefer to get a chance to review a
>> major change to the core RDMA midlayer rather than having you merge it
>> through your tree.  So yes I do object.  Please give me a chance to
>> review and merge it.  I am working on that this week.
>>
>
> Great.  We'll be looking for a response by the end of the week.
>
> Otherwise if you end up not having time, we'd still like to move forward for v3.15 given the amount of review the series has already gotten on the list.
>
> Thank you,
>
> --nab
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: linux rdma 3.14 merge plans

[Devesh Sharma]

Thanks Roland,
My response is inline below.

-Regards
 Devesh
-----Original Message-----
From: roland@purestorage.com [mailto:roland@purestorage.com] On Behalf Of Roland Dreier
Sent: Saturday, March 08, 2014 1:02 AM
To: Devesh Sharma
Cc: Nicholas A. Bellinger; Or Gerlitz; Hefty Sean; linux-rdma; Martin K. Petersen; target-devel; Sagi Grimberg; linux-kernel
Subject: Re: linux rdma 3.14 merge plans

Sure, no problem.

Do you have a git tree with the latest versions of all the changes you want for 3.15 in a branch?  That would be helpful as I catch up on applying things, so that I don't miss anything.
[DS]: Yes, I do have a cloned copy of your git tree migrated to for-next branch, with all the patches we have submitted to linux-rdma (patch series: [PATCH for-next 00/17] ocrdma driver code sync-up). However, it is on my local server.
In case of urgency (and if OFED EWG permits me) I can set it up on benay dot openfabrics dot org, for now and give a link to pull.

If you don't have one, taking a little time to set one up on github or wherever would be nice.  You can base your set of changes on Linus's latest tree.
[DS]: Currently we do not have such tree. I will have to discuss within my organization on how to go about this request. 

Thanks!
  Roland

On Thu, Mar 6, 2014 at 9:07 PM, Devesh Sharma <Devesh.Sharma@emulex.com> wrote:
> Hi Roland,
>
> Is it okay to send next series of patches even if previous series is not accepted yet in your tree? Off-course I will cut patches on top of previous series of patches.
>
> -Regards
>  Devesh
>
> -----Original Message-----
> From: linux-rdma-owner@vger.kernel.org 
> [mailto:linux-rdma-owner@vger.kernel.org] On Behalf Of Nicholas A. 
> Bellinger
> Sent: Thursday, March 06, 2014 12:34 AM
> To: Roland Dreier
> Cc: Or Gerlitz; Hefty Sean; linux-rdma; Martin K. Petersen; 
> target-devel; Sagi Grimberg; linux-kernel
> Subject: Re: linux rdma 3.14 merge plans
>
> On Wed, 2014-03-05 at 07:18 -0800, Roland Dreier wrote:
>> On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger 
>> <nab@linux-iscsi.org> wrote:
>> > That all said, do you have an objection wrt taking this bits 
>> > through target-pending..?  Given the dependencies involved, that 
>> > would seem the most logical path to take.
>>
>> Perhaps not surprisingly, I would prefer to get a chance to review a 
>> major change to the core RDMA midlayer rather than having you merge 
>> it through your tree.  So yes I do object.  Please give me a chance 
>> to review and merge it.  I am working on that this week.
>>
>
> Great.  We'll be looking for a response by the end of the week.
>
> Otherwise if you end up not having time, we'd still like to move forward for v3.15 given the amount of review the series has already gotten on the list.
>
> Thank you,
>
> --nab
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" 
> in the body of a message to majordomo@vger.kernel.org More majordomo 
> info at  http://vger.kernel.org/majordomo-info.html

Re: linux rdma 3.14 merge plans

[Or Gerlitz]

On Wed, Jan 22, 2014, Sagi Grimberg <sagig@dev.mellanox.co.il> wrote:
> On 1/22/2014 2:43 AM, Roland Dreier wrote:
>> On Tue, Jan 21, 2014, Or Gerlitz <or.gerlitz@gmail.com> wrote:

>>> Roland, ping! the signature patches were posted > three months ago. We
>>> deserve a response from the maintainer that goes beyond "I need to
>>> think on that". Responsiveness was stated by Linus to be the #1 requirement
>>> from kernel maintainers.

> Hi Roland, I'll try to respond here. removing LKML and adding Linux-scsi.

Sorry, it seems we will not getting responses unless coping LKML, so
lets do that again -- Roland, below is the detailed response Sagi
wrote you following your "adds a bunch of complexity to support a
feature no one really cares about" comment, can we get you to respond
on that?

>> Or, I'm not sure what response you're after from me.  Linus has also
>> said that maintainers should say "no" a lot more
>> (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I
>> won't merge this patch set, since it adds a bunch of complexity to
>> support a feature no one really cares about."

> 1. I disagree about no-one cares about DIF/DIX. We are witnessing growing
> interests in this especially for RDMA.
> 2. We put a lot of efforts to avoid complexity here and plug-in as simple as
> possible.
> Application that will choose to use DIF will implement only 3 steps:
> a. allocate signature enabled MR.
> b. register signature enabled MR with DIF attributes (via post_send) and
> then do RDMA.
> c. check MR status after transaction is completed (_lightweight_ verb that
> can be called from interrupt context).

>> Is that it?  (And yes I am skeptical about this stuff -- I work at an enterprise
>> storage company and even here it's hard to find anyone who cares about
>> DIF/DIX, especially offload features that stop it from being end-to-end)


> 1. RDMA verbs are _NOT_ stopping DIF from being end-to-end.
> OS (or SCSI in our specific case) passes LLD 2 scatterlists: data {block1,
> block2, block3,...}, and protection {DIF1, DIF2, DIF3}.
> LLD is required to verify the data integrity (block guards) and to
> interleave over the wire {block1, DIF1, block2, DIF2....}.
> You must support that in HW, you rather iSER/SRP will use giant copy's to
> interleave by itself? or in case OS asked LLD
> to INSERT DIF iSER/SRP will compute CRC for each data-block? RDMA storage
> ULPs are transports - they should have no business with
> data processing.

> 2. HW DIF offload also gives you protection across the PCI. the
> data-validation is done (hopefully offloaded) also
> when data+protection are written to the back-end device. end-to-end is
> preserved.

> 3. SAS & FC have T10-PI offload. This is just adding RDMA into the game.
> With this set of verbs iSER, SRP, FCoE Initiators and targets will be able
> to support T10-PI.


>> I'm sure you're not expecting me to say, "Sure, I'll merge it without
>> understanding the problem it's solving

> Problem: T10-PI offload support for RDMA based initiators. Supporting
> end-to-end data integrity while sustaining high RDMA performance.


>>   or how it's doing that,"

> How it's doing that:
> - We introduce a new type of memory region that posses protection attributes
> suited for data integrity offload.
> - We Introduce a new fast registration method that can bind all the relevant
> info for verify/generate of protection information:
>   * describe if/how to interleave data with protection.
>   * describe what method of data integrity is used (DIF type X, CRC, XOR...)
> and the seeds that HW should start calculation from.
>   * describe how to verify the data.
> - We Introduce a new lightweight check of the data-integrity status to check
> if there were any integrity errors and get information on them.

> Note: We made MR allocation routine generic enough to lay a framework to
> unite all MR allocation
> methods (get_dma_mr, alloc_fast_reg_mr, reg_phys, reg_user_mr, fmrs, and
> probably more in the future...).
> We defined ib_create_mr that can actually get mr_init_attr which can be
> easily extended as opposed to the specific calls exists today.
> So I would say this even reduces complexity.

> Hope this helps,
> Sagi.