git send-email friendly smtp provider anyone?

git send-email friendly smtp provider anyone?

[Noralf Trønnes]

Hi

A couple of years ago my email provider blocked me from using git
send-email with their smtp server. So I switched to the one my ISP
provides. Now my ISP have outsourced their email service so the first 3
emails gets through and the rest looks like it ends up in a tar pit or
something, 18 hours later and 5 of 7 emails have gotten through. I have
asked them about this, but I fear the answer will be this is not
supported since they now don't have the service in-house anymore. I'm
waiting for a reply.

Today I tried sendinblue.com since they have a free plan, but they
insert <br> in the emails so that didn't work out. They also have some
kind of queue, after 1 hour 6 of 7 emails have gotten through.

Does anyone have an smtp provider to recommend that works with git
send-email and that sends out all the emails at once?

I have a patchset that I want to send out.

Noralf.

Re: git send-email friendly smtp provider anyone?

[Simon Ser]

I think you can apply for a linux.dev mailbox [1].

[1]: https://korg.docs.kernel.org/linuxdev.html

Re: git send-email friendly smtp provider anyone?

[Noralf Trønnes]

Den 21.11.2022 14.33, skrev Simon Ser:
> I think you can apply for a linux.dev mailbox [1].
> 

Yeah you're right, I didn't know about that possibility.
But it depends on whether or not I can just use their smtp server and
keep my current email address. This looks like what's the problem with
my current ISP, I need to use the email account I have in their email
service (that I've never used) for sending through their smtp server,
but I want to send From: another email address.

Noralf.

> [1]: https://korg.docs.kernel.org/linuxdev.html

Re: git send-email friendly smtp provider anyone?

[Simon Ser]

On Monday, November 21st, 2022 at 17:52, Noralf Trønnes <noralf@tronnes.org> wrote:

> Den 21.11.2022 14.33, skrev Simon Ser:
> 
> > I think you can apply for a linux.dev mailbox 1.
> 
> Yeah you're right, I didn't know about that possibility.
> But it depends on whether or not I can just use their smtp server and
> keep my current email address. This looks like what's the problem with
> my current ISP, I need to use the email account I have in their email
> service (that I've never used) for sending through their smtp server,
> but I want to send From: another email address.

That's not possible. It breaks DKIM, so your emails will end up in Spam
folders or be rejected. You need to use the SMTP server tied to your
email address.

Re: git send-email friendly smtp provider anyone?

[Simon Ser]

On Monday, November 21st, 2022 at 18:02, Simon Ser <contact@emersion.fr> wrote:

> On Monday, November 21st, 2022 at 17:52, Noralf Trønnes noralf@tronnes.org wrote:
> 
> > Den 21.11.2022 14.33, skrev Simon Ser:
> > 
> > > I think you can apply for a linux.dev mailbox 1.
> > 
> > Yeah you're right, I didn't know about that possibility.
> > But it depends on whether or not I can just use their smtp server and
> > keep my current email address. This looks like what's the problem with
> > my current ISP, I need to use the email account I have in their email
> > service (that I've never used) for sending through their smtp server,
> > but I want to send From: another email address.
> 
> That's not possible. It breaks DKIM, so your emails will end up in Spam
> folders or be rejected. You need to use the SMTP server tied to your
> email address.

That said, you can send patches from an email address different from
the one in your patches. IOW, you can send patches committed by
<noralf@tronnes.org> from any email account.

The From in the email header won't match the commit, but the very first
line of the patch will hold that information.

Re: git send-email friendly smtp provider anyone?

[Noralf Trønnes]

Den 21.11.2022 18.06, skrev Simon Ser:
> On Monday, November 21st, 2022 at 18:02, Simon Ser <contact@emersion.fr> wrote:
> 
>> On Monday, November 21st, 2022 at 17:52, Noralf Trønnes noralf@tronnes.org wrote:
>>
>>> Den 21.11.2022 14.33, skrev Simon Ser:
>>>
>>>> I think you can apply for a linux.dev mailbox 1.
>>>
>>> Yeah you're right, I didn't know about that possibility.
>>> But it depends on whether or not I can just use their smtp server and
>>> keep my current email address. This looks like what's the problem with
>>> my current ISP, I need to use the email account I have in their email
>>> service (that I've never used) for sending through their smtp server,
>>> but I want to send From: another email address.
>>
>> That's not possible. It breaks DKIM, so your emails will end up in Spam
>> folders or be rejected. You need to use the SMTP server tied to your
>> email address.
> 
> That said, you can send patches from an email address different from
> the one in your patches. IOW, you can send patches committed by
> <noralf@tronnes.org> from any email account.
> 
> The From in the email header won't match the commit, but the very first
> line of the patch will hold that information.

Thanks that was useful information. I've seen the DKIM abbr. but haven't
looked into the meaning of it.

I tried:

git send-email --from=noralf.tronnes@altiboxmail.no
--reply=noralf@tronnes.org

and now I'm getting 'pass' in the Authentication-Results field, so
that's progress. I'm still not getting all the emails through, so I
still have that problem, I'll have to wait and see what the ISP can tell me.

But this means that a linux.dev mailbox is an option for me should my
ISP be a blocker.

Noralf.

Re: git send-email friendly smtp provider anyone?

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 1153 bytes --]

On Mon, Nov 21, 2022 at 12:48:52PM +0100, Noralf Trønnes wrote:
> A couple of years ago my email provider blocked me from using git
> send-email with their smtp server. So I switched to the one my ISP
> provides. Now my ISP have outsourced their email service so the first 3
> emails gets through and the rest looks like it ends up in a tar pit or
> something, 18 hours later and 5 of 7 emails have gotten through. I have
> asked them about this, but I fear the answer will be this is not
> supported since they now don't have the service in-house anymore. I'm
> waiting for a reply.
> 
> Today I tried sendinblue.com since they have a free plan, but they
> insert <br> in the emails so that didn't work out. They also have some
> kind of queue, after 1 hour 6 of 7 emails have gotten through.
> 
> Does anyone have an smtp provider to recommend that works with git
> send-email and that sends out all the emails at once?

I'm using fastmail and am very happy about it so far.

Otherwise, you might consider using:
https://b4.docs.kernel.org/en/latest/contributor/send.html#authenticating-with-the-web-submission-endpoint

Maxime

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: git send-email friendly smtp provider anyone?

[Noralf Trønnes]

Den 21.11.2022 16.19, skrev Maxime Ripard:
> On Mon, Nov 21, 2022 at 12:48:52PM +0100, Noralf Trønnes wrote:
>> A couple of years ago my email provider blocked me from using git
>> send-email with their smtp server. So I switched to the one my ISP
>> provides. Now my ISP have outsourced their email service so the first 3
>> emails gets through and the rest looks like it ends up in a tar pit or
>> something, 18 hours later and 5 of 7 emails have gotten through. I have
>> asked them about this, but I fear the answer will be this is not
>> supported since they now don't have the service in-house anymore. I'm
>> waiting for a reply.
>>
>> Today I tried sendinblue.com since they have a free plan, but they
>> insert <br> in the emails so that didn't work out. They also have some
>> kind of queue, after 1 hour 6 of 7 emails have gotten through.
>>
>> Does anyone have an smtp provider to recommend that works with git
>> send-email and that sends out all the emails at once?
> 
> I'm using fastmail and am very happy about it so far.
> 
> Otherwise, you might consider using:
> https://b4.docs.kernel.org/en/latest/contributor/send.html#authenticating-with-the-web-submission-endpoint
> 

That's an interesting option. I did briefly look at b4 a few months back
but it looked like it was under heavy development so I figured I'd wait
before trying it out. I think I'll give b4 a spin to see how it works, I
wonder how it handles patch changelogs.

Noralf.

Re: git send-email friendly smtp provider anyone?

[Konstantin Ryabitsev]

On Mon, Nov 21, 2022 at 07:13:28PM +0100, Noralf Trønnes wrote:
> > Otherwise, you might consider using:
> > https://b4.docs.kernel.org/en/latest/contributor/send.html#authenticating-with-the-web-submission-endpoint
> > 
> 
> That's an interesting option. I did briefly look at b4 a few months back
> but it looked like it was under heavy development so I figured I'd wait
> before trying it out. I think I'll give b4 a spin to see how it works, I
> wonder how it handles patch changelogs.

I'd be happy to help set this up for you -- to date, this service hasn't been
used beyond a few test posts.

-K

Re: git send-email friendly smtp provider anyone?

[Noralf Trønnes]

Den 22.11.2022 16.51, skrev Konstantin Ryabitsev:
> On Mon, Nov 21, 2022 at 07:13:28PM +0100, Noralf Trønnes wrote:
>>> Otherwise, you might consider using:
>>> https://b4.docs.kernel.org/en/latest/contributor/send.html#authenticating-with-the-web-submission-endpoint
>>>
>>
>> That's an interesting option. I did briefly look at b4 a few months back
>> but it looked like it was under heavy development so I figured I'd wait
>> before trying it out. I think I'll give b4 a spin to see how it works, I
>> wonder how it handles patch changelogs.
> 
> I'd be happy to help set this up for you -- to date, this service hasn't been
> used beyond a few test posts.
> 

Ok, I'll give it a try.

I have now prepared the patchset, generated a key and can now do:
b4 send -o

The first thing that strikes me is that everyone mentioned in one of the
patches get the entire patchset, even stable@vger.kernel.org (cc'ed in a
fixes patch). The first patch touches a core file and as a result a few
drivers, so I've cc'ed the driver maintainers in that patch, but now
they get the entire patchset where 5 of 6 patches is about a driver that
I maintain. So from their point of view, they see a patchset about a
driver they don't care about and a patch touching a core file, but from
the subject it's not apparent that it touches their driver. I'm afraid
that this might result in none of them looking at that patch. In this
particular case it's not that important, but in another case it might be.

As for the setting up the web endpoint, should I just follow the b4 docs
on that?

I use b4 version 0.10.1, is that recent enough?

Noralf.

Re: git send-email friendly smtp provider anyone?

[Konstantin Ryabitsev]

On Tue, Nov 22, 2022 at 06:42:19PM +0100, Noralf Trønnes wrote:
> The first thing that strikes me is that everyone mentioned in one of the
> patches get the entire patchset, even stable@vger.kernel.org (cc'ed in a
> fixes patch). The first patch touches a core file and as a result a few
> drivers, so I've cc'ed the driver maintainers in that patch, but now
> they get the entire patchset where 5 of 6 patches is about a driver that
> I maintain. So from their point of view, they see a patchset about a
> driver they don't care about and a patch touching a core file, but from
> the subject it's not apparent that it touches their driver. I'm afraid
> that this might result in none of them looking at that patch. In this
> particular case it's not that important, but in another case it might be.

I did some (unscientific) polling among kernel maintainers and, by a vast
margin, they always prefer to receive the entire series instead of
cherry-picked patches -- having the entire series helps provide important
context for the change they are looking at.

So, this is deliberate and, for now at least, not configurable. Unless you're
sending 100+ patch series, I doubt anyone will have any problem with receiving
the whole series instead of individual patches.

> As for the setting up the web endpoint, should I just follow the b4 docs
> on that?
> 
> I use b4 version 0.10.1, is that recent enough?

Yes. There will be a 0.10.2 in the near future, but the incoming fixes
shouldn't make much difference for the b4 send code.

-K

Re: git send-email friendly smtp provider anyone?

[Noralf Trønnes]

Den 22.11.2022 19.50, skrev Konstantin Ryabitsev:
> On Tue, Nov 22, 2022 at 06:42:19PM +0100, Noralf Trønnes wrote:
>> The first thing that strikes me is that everyone mentioned in one of the
>> patches get the entire patchset, even stable@vger.kernel.org (cc'ed in a
>> fixes patch). The first patch touches a core file and as a result a few
>> drivers, so I've cc'ed the driver maintainers in that patch, but now
>> they get the entire patchset where 5 of 6 patches is about a driver that
>> I maintain. So from their point of view, they see a patchset about a
>> driver they don't care about and a patch touching a core file, but from
>> the subject it's not apparent that it touches their driver. I'm afraid
>> that this might result in none of them looking at that patch. In this
>> particular case it's not that important, but in another case it might be.
> 
> I did some (unscientific) polling among kernel maintainers and, by a vast
> margin, they always prefer to receive the entire series instead of
> cherry-picked patches -- having the entire series helps provide important
> context for the change they are looking at.
> 
> So, this is deliberate and, for now at least, not configurable. Unless you're
> sending 100+ patch series, I doubt anyone will have any problem with receiving
> the whole series instead of individual patches.
> 
>> As for the setting up the web endpoint, should I just follow the b4 docs
>> on that?
>>
>> I use b4 version 0.10.1, is that recent enough?
> 
> Yes. There will be a 0.10.2 in the near future, but the incoming fixes
> shouldn't make much difference for the b4 send code.
> 

This is what I got:

$ b4 send --web-auth-verify <challenge string from email>
Signing challenge
Submitting verification to https://lkml.kernel.org/_b4_submit
Traceback (most recent call last):
  File "/home/pi/.local/bin/b4", line 8, in <module>
    sys.exit(cmd())
  File "/home/pi/.local/lib/python3.10/site-packages/b4/command.py",
line 341, in cmd
    cmdargs.func(cmdargs)
  File "/home/pi/.local/lib/python3.10/site-packages/b4/command.py",
line 86, in cmd_send
    b4.ez.cmd_send(cmdargs)
  File "/home/pi/.local/lib/python3.10/site-packages/b4/ez.py", line
1102, in cmd_send
    auth_verify(cmdargs)
  File "/home/pi/.local/lib/python3.10/site-packages/b4/ez.py", line
188, in auth_verify
    res = ses.post(endpoint, json=req)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 590,
in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 528,
in request
    prep = self.prepare_request(req)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 456,
in prepare_request
    p.prepare(
  File "/usr/lib/python3/dist-packages/requests/models.py", line 319, in
prepare
    self.prepare_body(data, files, json)
  File "/usr/lib/python3/dist-packages/requests/models.py", line 469, in
prepare_body
    body = complexjson.dumps(json)
  File "/usr/lib/python3.10/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

$ python3 --version
Python 3.10.6

Turning on debug output didn't add much:

$ b4 -d send --web-auth-verify 7ad470b4-f531-4632-8093-738d4d3e5d88
Running git --no-pager rev-parse --show-toplevel
Running git --no-pager config -z --get-regexp b4\..*
Running git --no-pager config -z --get-regexp gpg\..*
Running git --no-pager config -z --get-regexp user\..*
Signing challenge
Submitting verification to https://lkml.kernel.org/_b4_submit
Traceback (most recent call last):
  File "/home/pi/.local/bin/b4", line 8, in <module>
    sys.exit(cmd())
  File "/home/pi/.local/lib/python3.10/site-packages/b4/command.py",
line 341, in cmd
    cmdargs.func(cmdargs)
  File "/home/pi/.local/lib/python3.10/site-packages/b4/command.py",
line 86, in cmd_send
    b4.ez.cmd_send(cmdargs)
  File "/home/pi/.local/lib/python3.10/site-packages/b4/ez.py", line
1102, in cmd_send
    auth_verify(cmdargs)
  File "/home/pi/.local/lib/python3.10/site-packages/b4/ez.py", line
188, in auth_verify
    res = ses.post(endpoint, json=req)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 590,
in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 528,
in request
    prep = self.prepare_request(req)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 456,
in prepare_request
    p.prepare(
  File "/usr/lib/python3/dist-packages/requests/models.py", line 319, in
prepare
    self.prepare_body(data, files, json)
  File "/usr/lib/python3/dist-packages/requests/models.py", line 469, in
prepare_body
    body = complexjson.dumps(json)
  File "/usr/lib/python3.10/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable


Noralf.

Re: git send-email friendly smtp provider anyone?

[Noralf Trønnes]

Den 22.11.2022 20.22, skrev Noralf Trønnes:
> 
> 
> Den 22.11.2022 19.50, skrev Konstantin Ryabitsev:
>> On Tue, Nov 22, 2022 at 06:42:19PM +0100, Noralf Trønnes wrote:
>>> The first thing that strikes me is that everyone mentioned in one of the
>>> patches get the entire patchset, even stable@vger.kernel.org (cc'ed in a
>>> fixes patch). The first patch touches a core file and as a result a few
>>> drivers, so I've cc'ed the driver maintainers in that patch, but now
>>> they get the entire patchset where 5 of 6 patches is about a driver that
>>> I maintain. So from their point of view, they see a patchset about a
>>> driver they don't care about and a patch touching a core file, but from
>>> the subject it's not apparent that it touches their driver. I'm afraid
>>> that this might result in none of them looking at that patch. In this
>>> particular case it's not that important, but in another case it might be.
>>
>> I did some (unscientific) polling among kernel maintainers and, by a vast
>> margin, they always prefer to receive the entire series instead of
>> cherry-picked patches -- having the entire series helps provide important
>> context for the change they are looking at.
>>
>> So, this is deliberate and, for now at least, not configurable. Unless you're
>> sending 100+ patch series, I doubt anyone will have any problem with receiving
>> the whole series instead of individual patches.
>>
>>> As for the setting up the web endpoint, should I just follow the b4 docs
>>> on that?
>>>
>>> I use b4 version 0.10.1, is that recent enough?
>>
>> Yes. There will be a 0.10.2 in the near future, but the incoming fixes
>> shouldn't make much difference for the b4 send code.
>>
> 
> This is what I got:
> 
> $ b4 send --web-auth-verify <challenge string from email>
> Signing challenge
> Submitting verification to https://lkml.kernel.org/_b4_submit
> Traceback (most recent call last):
>   File "/home/pi/.local/bin/b4", line 8, in <module>
>     sys.exit(cmd())
>   File "/home/pi/.local/lib/python3.10/site-packages/b4/command.py",
> line 341, in cmd
>     cmdargs.func(cmdargs)
>   File "/home/pi/.local/lib/python3.10/site-packages/b4/command.py",
> line 86, in cmd_send
>     b4.ez.cmd_send(cmdargs)
>   File "/home/pi/.local/lib/python3.10/site-packages/b4/ez.py", line
> 1102, in cmd_send
>     auth_verify(cmdargs)
>   File "/home/pi/.local/lib/python3.10/site-packages/b4/ez.py", line
> 188, in auth_verify
>     res = ses.post(endpoint, json=req)
>   File "/usr/lib/python3/dist-packages/requests/sessions.py", line 590,
> in post
>     return self.request('POST', url, data=data, json=json, **kwargs)
>   File "/usr/lib/python3/dist-packages/requests/sessions.py", line 528,
> in request
>     prep = self.prepare_request(req)
>   File "/usr/lib/python3/dist-packages/requests/sessions.py", line 456,
> in prepare_request
>     p.prepare(
>   File "/usr/lib/python3/dist-packages/requests/models.py", line 319, in
> prepare
>     self.prepare_body(data, files, json)
>   File "/usr/lib/python3/dist-packages/requests/models.py", line 469, in
> prepare_body
>     body = complexjson.dumps(json)
>   File "/usr/lib/python3.10/json/__init__.py", line 231, in dumps
>     return _default_encoder.encode(obj)
>   File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
>     chunks = self.iterencode(o, _one_shot=True)
>   File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
>     return _iterencode(o, 0)
>   File "/usr/lib/python3.10/json/encoder.py", line 179, in default
>     raise TypeError(f'Object of type {o.__class__.__name__} '
> TypeError: Object of type bytes is not JSON serializable
> 

Konstantin found a workaround, so I was able to push the patches.

Here's the result if anyone is interested in seeing the result of using
b4 and the web endpoint:
https://lore.kernel.org/dri-devel/20221122-gud-shadow-plane-v1-0-9de3afa3383e@tronnes.org/

Patchwork gave me a new submitter ID:
https://patchwork.freedesktop.org/series/111222/

Noralf.

Re: git send-email friendly smtp provider anyone?

[Konstantin Ryabitsev]

On Tue, Nov 22, 2022 at 10:10:47PM +0100, Noralf Trønnes wrote:
> Konstantin found a workaround, so I was able to push the patches.

Yes, this uncovered quite a few bugs -- which is excellent for me, not so
excellent for you. :)

> Here's the result if anyone is interested in seeing the result of using
> b4 and the web endpoint:
> https://lore.kernel.org/dri-devel/20221122-gud-shadow-plane-v1-0-9de3afa3383e@tronnes.org/
> 
> Patchwork gave me a new submitter ID:
> https://patchwork.freedesktop.org/series/111222/

Oooh, I see that patchwork is still not doing the right thing with
X-Original-From. It will only do the substitution when the From email address
is the same as the email address of the list.

https://github.com/getpatchwork/patchwork/blob/main/patchwork/parser.py#L437

There's unfortunately no fix for this that I can do on my end. :(

-K