API email notifications SMTPAuthenticationError

[galviset]

This is an echo to this issue : lifewatch-eric/ontologies_api#9

It is also the case for EarthPortal, so this bug might be common to most appliances.
Emails are working fine through the Feedback form (which is done from the UI), but does not in the case of API notifications despite credentials being identical in both UI and API configuration files.

Logs :

2023-10-25 12:55:16 - Net::SMTPAuthenticationError - 530 5.7.1 Client was not authenticated
:
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/net-smtp-0.3.3/lib/net/smtp.rb:1076:in `check_response'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/net-smtp-0.3.3/lib/net/smtp.rb:1044:in `getok'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/net-smtp-0.3.3/lib/net/smtp.rb:945:in `mailfrom'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/net-smtp-0.3.3/lib/net/smtp.rb:767:in `send_message'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/mail-2.8.1/lib/mail/network/delivery_methods/smtp_connection.rb:53:in `deliver!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/mail-2.8.1/lib/mail/network/delivery_methods/smtp.rb:101:in `block in deliver!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/net-smtp-0.3.3/lib/net/smtp.rb:612:in `start'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/mail-2.8.1/lib/mail/network/delivery_methods/smtp.rb:109:in `start_smtp_session'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/mail-2.8.1/lib/mail/network/delivery_methods/smtp.rb:100:in `deliver!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/mail-2.8.1/lib/mail/message.rb:269:in `deliver!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/pony-1.13.1/lib/pony.rb:177:in `deliver'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/pony-1.13.1/lib/pony.rb:166:in `mail'
        /srv/ontoportal/.bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-e9b708c40b2b/lib/ontologies_linked_data/utils/notifier.rb:21:in `notify'
        /srv/ontoportal/.bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-e9b708c40b2b/lib/ontologies_linked_data/utils/notifier.rb:58:in `notify_mails_grouped'
        /srv/ontoportal/.bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-e9b708c40b2b/lib/ontologies_linked_data/utils/notifier.rb:48:in `block in notify_mails_separately'
        /srv/ontoportal/.bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-e9b708c40b2b/lib/ontologies_linked_data/utils/notifier.rb:47:in `each'
        /srv/ontoportal/.bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-e9b708c40b2b/lib/ontologies_linked_data/utils/notifier.rb:47:in `notify_mails_separately'
        /srv/ontoportal/.bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-e9b708c40b2b/lib/ontologies_linked_data/utils/notifications.rb:113:in `reset_password'
        /srv/ontoportal/ontologies_api/releases/20230808103646/controllers/users_controller.rb:29:in `block (2 levels) in <class:UsersController>'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1611:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1611:in `block in compile!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:975:in `block (3 levels) in route!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:994:in `route_eval'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:975:in `block (2 levels) in route!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1015:in `block in process_route'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1013:in `catch'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1013:in `process_route'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:973:in `block in route!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:972:in `each'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:972:in `route!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1085:in `block in dispatch!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `block in invoke'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `catch'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `invoke'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:907:in `call!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:895:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-cache-1.13.0/lib/rack/cache/context.rb:141:in `forward'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-cache-1.13.0/lib/rack/cache/context.rb:148:in `pass'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-cache-1.13.0/lib/rack/cache/context.rb:160:in `invalidate'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-cache-1.13.0/lib/rack/cache/context.rb:75:in `call!'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-cache-1.13.0/lib/rack/cache/context.rb:52:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-e9b708c40b2b/lib/ontologies_linked_data/security/access_denied_middleware.rb:10:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-e9b708c40b2b/lib/ontologies_linked_data/security/authorization.rb:45:in `call'
        /srv/ontoportal/ontologies_api/releases/20230808103646/lib/rack/param_translator.rb:47:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/bundler/gems/rack-post-body-to-params-0fd30e710386/lib/rack/post-body-to-params.rb:144:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-accept-0.4.5/lib/rack/accept/context.rb:22:in `call'
        /srv/ontoportal/ontologies_api/releases/20230808103646/lib/rack/slice_detection.rb:36:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-cors-1.0.6/lib/rack/cors.rb:98:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-cors-1.0.6/lib/rack/cors.rb:98:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-1.6.13/lib/rack/static.rb:124:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-1.6.13/lib/rack/commonlogger.rb:33:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:219:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-protection-1.5.5/lib/rack/protection/xss_header.rb:18:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-protection-1.5.5/lib/rack/protection/json_csrf.rb:18:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-protection-1.5.5/lib/rack/protection/base.rb:49:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-protection-1.5.5/lib/rack/protection/base.rb:49:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-protection-1.5.5/lib/rack/protection/frame_options.rb:31:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-1.6.13/lib/rack/logger.rb:15:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-1.6.13/lib/rack/commonlogger.rb:33:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:219:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:212:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-1.6.13/lib/rack/head.rb:13:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/rack-1.6.13/lib/rack/methodoverride.rb:22:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:182:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:2013:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in `block in call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1787:in `synchronize'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in `call'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/unicorn-6.1.0/lib/unicorn/http_server.rb:634:in `process_client'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/unicorn-worker-killer-0.4.5/lib/unicorn/worker_killer.rb:53:in `process_client'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/unicorn-worker-killer-0.4.5/lib/unicorn/worker_killer.rb:93:in `process_client'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/unicorn-6.1.0/lib/unicorn/http_server.rb:739:in `worker_loop'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/unicorn-6.1.0/lib/unicorn/http_server.rb:547:in `spawn_missing_workers'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/unicorn-6.1.0/lib/unicorn/http_server.rb:143:in `start'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/unicorn-6.1.0/bin/unicorn:128:in `<top (required)>'
        /srv/ontoportal/.bundle/ruby/2.7.0/bin/unicorn:23:in `load'
        /srv/ontoportal/.bundle/ruby/2.7.0/bin/unicorn:23:in `<top (required)>'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/cli/exec.rb:58:in `load'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/cli/exec.rb:58:in `kernel_load'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/cli/exec.rb:23:in `run'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/cli.rb:486:in `exec'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/cli.rb:31:in `dispatch'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/cli.rb:25:in `start'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/exe/bundle:48:in `block in <top (required)>'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/lib/bundler/friendly_errors.rb:120:in `with_friendly_errors'
        /srv/ontoportal/.bundle/ruby/2.7.0/gems/bundler-2.3.23/exe/bundle:36:in `<top (required)>'
        /usr/local/rbenv/versions/2.7.6/bin/bundle:25:in `load'
        /usr/local/rbenv/versions/2.7.6/bin/bundle:25:in `<main>'

API email configuration:

# Email notifications
    config.enable_notifications   = true
    config.email_sender           = 'support@earthportal.eu' # Default sender for emails
    config.email_override         = 'support@earthportal.eu' # all email gets sent here. Disable with email_override_disable.
    config.email_disable_override = true
    config.smtp_host              = 'ssl0.ovh.net'
    config.smtp_port              = '587'
    config.smtp_auth_type         = :plain # :none, :plain, :login, :cram_md5
    config.smtp_user              = "support@earthportal.eu"
    config.smtp_password          = "<password>"
    config.smtp_domain            = 'earthportal.eu'
    config.smtp_tls               = true

[syphax-bouazzouni]

Some additional info:

• The UI uses ActionMailer to send emails, and the API uses the gem pony.
• In AgroPortal we don't have smtp_user and smtp_password and it works and the API

So I hypothesize that Pony may not be well configured when we use login authentication type.

@alexskr any idea about this, in Bioportal is does work, do you use login authentication for your SMTP server?

Ping @gturrisi-lifewatch FYI

[alexskr]

we don't use authenticated smtp for Bioportal so we have not run into this problem on our end yet.

[alexskr]

Bioportal doesn't have config.smtp_tls configuration option. Is this a new config parameter that Agroportal/EarthPortal added?

[syphax-bouazzouni]

I don't know where does come smtp_tls, @galviset ? the correct config is enable_starttls_auto. But even with this change, I don't think it works.

here are all our mailing configs:

https://github.com/ontoportal-lirmm/ontologies_linked_data/blob/master/lib/ontologies_linked_data/utils/notifier.rb#L21-L30

https://github.com/ontoportal-lirmm/ontologies_linked_data/blob/master/lib/ontologies_linked_data/utils/notifier.rb#L106-L122

[galviset]

It is a mistake indeed but as you said restoring it to enable_starttls_auto does not influence the issue.

[syphax-bouazzouni]

Maybe we need to add an extra option, to add the TLS option in pony , we need to add unit tests to this.