I'm trying to read the README and figure out based on my subroutines, evaluator, surgeon import, and html data how to mesh it all together.
it's tough to know how to go from the yaml config to something that actually works. for example, here is what I have so far:
import { ScraperOperation } from './_base';
import * as request from 'request';
import * as surgeon from 'surgeon';
const URL = 'http://mysite.com';
export class Scraper extends ScraperOperation {
public scrape() {
this.readYaml(`test.yaml`) //read as string
.then(mainSiteInstructions => {
console.log(mainSiteInstructions);
request(URL, (err, res, body) => {
if(err || res.statusCode !== 200) return;
const x = surgeon.default({ evaluator: surgeon.cheerioEvaluator() });
x(mainSiteInstructions, body);
});
});
}
}
where the yaml file is:
alphabetLinks:
- select a {0, } | rp title | test /Rule Index/
I just seem to continuously get errors though.
I'm trying to read the README and figure out based on my subroutines, evaluator, surgeon import, and html data how to mesh it all together.
it's tough to know how to go from the yaml config to something that actually works. for example, here is what I have so far:
where the yaml file is:
I just seem to continuously get errors though.