[BUG]: Invoice pdf generation fails

[philipp1992]

Hi

using v1.17.4, when an invoice is finalized and payment processed, the file_url is null until i manually download the invoice using the Lago UI.
This works and the invoice is being downloaded. After that the file_url property is set to the correct url.
Without manually downloading the invoice, file_url is never set.

In the lago-worker i see this error:

5","lvl":"INFO","msg":"fail","ctx":{"class":"Invoices::GeneratePdfAndNotifyJob","jid":"861130f6a3d8fd3e0582eafc","elapsed":0.05}}
{"ts":"2025-01-10T19:27:11.004Z","pid":1,"tid":"7y05","lvl":"WARN","msg":"{"context":"Job raised exception","job":{"retry":0,"queue":"invoices","wrapped":"Invoices::GeneratePdfAndNotifyJob","args":[{"job_class":"Invoices::GeneratePdfAndNotifyJob","job_id":"5061a689-4833-480f-a4c2-a6b04198735b","provider_job_id":null,"queue_name":"invoices","priority":null,"arguments":[{"invoice":{"_aj_globalid":"gid://lago-api/Invoice/d3558088-8a53-40db-bd01-4564d1d794eb"},"email":false,"_aj_ruby2_keywords":["invoice","email"]}],"executions":0,"exception_executions":{},"locale":"en","timezone":"UTC","enqueued_at":"2025-01-10T19:27:10.949713189Z","scheduled_at":null,"tracing_info":{"membership":"membership/433a04d3-8bd8-4b90-8da3-2a1330f0647c"}}],"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","jid":"861130f6a3d8fd3e0582eafc","created_at":1736537230.949872,"enqueued_at":1736537230.950109}}"}
{"ts":"2025-01-10T19:27:11.005Z","pid":1,"tid":"7y05","lvl":"WARN","msg":"TypeError: Can't load the template file. Pass a string with a path or an object that responds to 'to_str', 'path' or 'to_path'"}
{"ts":"2025-01-10T19:27:11.007Z","pid":1,"tid":"7y05","lvl":"WARN","msg":""}

[jdenquin]

Hello @philipp1992

I'm not sure to understand to error, do you have more context maybe?
Are you using only one worker? (no dedicated PDF worker?)

[philipp1992]

Hello @philipp1992

I'm not sure to understand to error, do you have more context maybe?
Are you using only one worker? (no dedicated PDF worker?)

Using the Helm deployment. PDF generation fails with above error and the file url is empty

[jdenquin]

@philipp1992 do you use S3, GCS or the standard storage configuration?

[philipp1992]

i use the standard storage config

[jdenquin]

do you use minio or PersistentVolumeClaim options?

[philipp1992]

i use your helm chart.

lago-billing-worker has

spec:
  volumes:
    - name: lago-storage-data
      persistentVolumeClaim:
        claimName: lago-storage-data

.....
volumeMounts:
- name: lago-storage-data
mountPath: /app/storage

these are my helm values.

lago:
  apiUrl: https://billing-api.dev.xxx.de
  frontUrl: https://billing-ui.dev.xxx.de
  api:
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
  eventsWorker:
    livenessProbe:
      initialDelaySeconds: 300000
      periodSeconds: 1000
      timeoutSeconds: 5000
      successThreshold: 1
      failureThreshold: 30001
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
  clockWorker:
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
  pdf: 
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
  webhookWorker:
    livenessProbe:
      initialDelaySeconds: 300000
      periodSeconds: 1000
      timeoutSeconds: 5000
      successThreshold: 1
      failureThreshold: 30000  
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
  billingWorker:
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
  worker:
    livenessProbe:
      initialDelaySeconds: 300000
      periodSeconds: 1000
      timeoutSeconds: 5000
      successThreshold: 1
      failureThreshold: 30000
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
  global:
    ingress:
      enabled: true
      annotations:
        cert-manager.io/cluster-issuer: letsencrypt-prod
      className: nginx
      frontHostname: billing-ui.dev.xxxx.de
      apiHostname: billing-api.dev.xxx.de

  postgresql:
    defaultStorageClass: hcloud-volumes

When i go to the lago UI, i can download the invoice and once i do that, this particular invoice also gets a proper file_url.

[jdenquin]

I think its because the PVC is configured as RWO (Read Write Once) and 2 different pods can't access it like we highlight here : https://github.com/getlago/lago-helm-charts/tree/main?tab=readme-ov-file#storage-recommendation

If you want to stay with PVC, you should tweak it to use RWM (depending on your cloud provider, it can be complex), I would recommend you to use S3, GCS or Minio

[philipp1992]

i doubt that this is the issue because then the other pod wouldnt even start. i have no other pods that would want to mount this volume.

Also why does it work when i manually download the PDF and also continue to work subsequently

[philipp1992]

cani migrate to s3 with an existing installation?

[philipp1992]

@jdenquin

[jdenquin]

yes but you will lose your PDF already generated, you may need to regenerate them