Skip to content

Sourcery refactored main branch #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Sourcery refactored main branch #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 5 additions & 11 deletions clean-data.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,17 @@ def clean_pinescript(script):
# Remove numbered lines dynamically
lines = [line for line in lines if not re.match(r"^\d+\s*$", line.strip())]

# Find the index of "Copy code" line
copy_code_index = -1
for i, line in enumerate(lines):
if line.strip() == 'Copy code':
copy_code_index = i
break

copy_code_index = next(
(i for i, line in enumerate(lines) if line.strip() == 'Copy code'), -1
)
Comment on lines -13 to +15
Copy link
Author

@sourcery-ai sourcery-ai bot Sep 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function clean_pinescript refactored with the following changes:

This removes the following comments ( why? ):

# Find the index of "Copy code" line

# Remove the last line if it contains the word "Expand" followed by parentheses and a number
if re.match(r"Expand \(\d+ lines\)", lines[-1].strip()):
lines = lines[:-1]

# Add comments to the lines preceding "Copy code"
comment_lines = []
for i in range(copy_code_index):
line = lines[i].strip()
if line:
if line := lines[i].strip():
comment_lines.append(f"// {line}")
lines = comment_lines + lines[copy_code_index + 1:]

Expand All @@ -34,8 +29,7 @@ def clean_pinescript(script):
cleaned_script = cleaned_script.replace('\n\n', '\n')
cleaned_script = cleaned_script.replace('=', ' = ')
cleaned_script = cleaned_script.replace(':', ': ')
cleaned_script = cleaned_script.replace(' ', ' ')
return cleaned_script
return cleaned_script.replace(' ', ' ')

# Specify the folder paths
input_folder = 'PineScripts'
Expand Down
2 changes: 1 addition & 1 deletion count.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ def format_size(size):
# Specify the directory path you want to count files in
directory_path = "PineScripts"

file_count = sum([len(files) for _, _, files in os.walk(directory_path)])
file_count = sum(len(files) for _, _, files in os.walk(directory_path))
Copy link
Author

@sourcery-ai sourcery-ai bot Sep 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 25-25 refactored with the following changes:

directory_size = get_directory_size(directory_path)

print(f"The number of files in '{directory_path}' is: {file_count}")
Expand Down
4 changes: 2 additions & 2 deletions fetch.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
page = 1

while True:
url = base_url + f'page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
url = f'{base_url}page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
Copy link
Author

@sourcery-ai sourcery-ai bot Sep 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 21-42 refactored with the following changes:

response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')

Expand All @@ -39,7 +39,7 @@

# Extract PineScript code
script_url = item.find('a', class_='tv-widget-idea__title').get('href')
script_full_url = 'https://www.tradingview.com' + script_url
script_full_url = f'https://www.tradingview.com{script_url}'

# Open the script page with Firefox
driver.get(script_full_url)
Expand Down
4 changes: 2 additions & 2 deletions fetch_editors-picks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
base_url = 'https://www.tradingview.com/scripts/editors-picks/'

async def download_page(session, page):
url = base_url + f'page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
url = f'{base_url}page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
Copy link
Author

@sourcery-ai sourcery-ai bot Sep 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function download_page refactored with the following changes:


async with session.get(url) as response:
content = await response.text()
Expand Down Expand Up @@ -60,7 +60,7 @@ async def download_page(session, page):

# Extract PineScript code
script_url = item.find('a', class_='tv-widget-idea__title').get('href')
script_full_url = 'https://www.tradingview.com' + script_url
script_full_url = f'https://www.tradingview.com{script_url}'

# Open the script page with Firefox
driver.get(script_full_url)
Expand Down
4 changes: 2 additions & 2 deletions fetch_indicators.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
base_url = 'https://www.tradingview.com/scripts/indicators/'

async def download_page(session, page):
url = base_url + f'page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
url = f'{base_url}page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
Copy link
Author

@sourcery-ai sourcery-ai bot Sep 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function download_page refactored with the following changes:


async with session.get(url) as response:
content = await response.text()
Expand Down Expand Up @@ -60,7 +60,7 @@ async def download_page(session, page):

# Extract PineScript code
script_url = item.find('a', class_='tv-widget-idea__title').get('href')
script_full_url = 'https://www.tradingview.com' + script_url
script_full_url = f'https://www.tradingview.com{script_url}'

# Open the script page with Firefox
driver.get(script_full_url)
Expand Down
4 changes: 2 additions & 2 deletions fetch_more.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
base_url = 'https://www.tradingview.com/scripts/'

def download_page(page):
url = base_url + f'page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
url = f'{base_url}page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
Copy link
Author

@sourcery-ai sourcery-ai bot Sep 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function download_page refactored with the following changes:

response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')

Expand All @@ -44,7 +44,7 @@ def download_page(page):

# Extract PineScript code
script_url = item.find('a', class_='tv-widget-idea__title').get('href')
script_full_url = 'https://www.tradingview.com' + script_url
script_full_url = f'https://www.tradingview.com{script_url}'

# Open the script page with Firefox
driver.get(script_full_url)
Expand Down
4 changes: 2 additions & 2 deletions fetch_scripts.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
base_url = 'https://www.tradingview.com/scripts/'

async def download_page(session, page):
url = base_url + f'page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
url = f'{base_url}page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
Copy link
Author

@sourcery-ai sourcery-ai bot Sep 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function download_page refactored with the following changes:


async with session.get(url) as response:
content = await response.text()
Expand Down Expand Up @@ -60,7 +60,7 @@ async def download_page(session, page):

# Extract PineScript code
script_url = item.find('a', class_='tv-widget-idea__title').get('href')
script_full_url = 'https://www.tradingview.com' + script_url
script_full_url = f'https://www.tradingview.com{script_url}'

# Open the script page with Firefox
driver.get(script_full_url)
Expand Down
4 changes: 2 additions & 2 deletions fetch_strategies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
base_url = 'https://www.tradingview.com/scripts/strategies/'

async def download_page(session, page):
url = base_url + f'page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
url = f'{base_url}page-{page}/?script_type=strategies&script_access=open&sort=month_popular&route_range=1'
Copy link
Author

@sourcery-ai sourcery-ai bot Sep 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function download_page refactored with the following changes:


async with session.get(url) as response:
content = await response.text()
Expand Down Expand Up @@ -60,7 +60,7 @@ async def download_page(session, page):

# Extract PineScript code
script_url = item.find('a', class_='tv-widget-idea__title').get('href')
script_full_url = 'https://www.tradingview.com' + script_url
script_full_url = f'https://www.tradingview.com{script_url}'

# Open the script page with Firefox
driver.get(script_full_url)
Expand Down