Skip to content

improve max length handling #733

@jordanxlau

Description

@jordanxlau

Bug description

Currently, the algorithm in textsplit.py will cut a sentence off at the max_length, even if this splits up a word. Instead, this algorithm should always preserve words and perhaps even preserve whole sentences or phrases.

How to reproduce the bug

# Sample code to reproduce the problem

Error messages and logs

# Error messages and logs here please

Environment

Current environment
# Please paste the output of `everyvoice --diagnostic` here
# EveryVoice Diagnostic information

More info

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions