Hoe programmeer je in Grimport?

Vraag ons om hulp bij het programmeren in Grimport TRANSLATE THIS PAGE WITHOUT TOUCHING THE CODES
Summary
  1. Grimport Language Documentation
    1. Introduction
    2. What is Grimport?
    3. Why use Grimport?
    4. Paradigms of language
  2. Installation of Grimport & Other Software
    1. How to get Grimport
    2. Other software to install
    3. Practicing software
  3. Variables & Syntax
    1. What is a variable?
    2. Escape sequence
    3. Arithmetic operators
    4. Relational operators
    5. Logical operators
    6. Conditional operators
    7. Lists
    8. Handling null values
    9. Scope rules
    10. Functions
    11. Optional arguments
    12. Default variables
    13. Comments
  4. Control Structures
    1. Conditional structures
    2. Looping structures
    3. Methods for iterating
    4. The break & continue statements
  5. Extract data from a page
    1. Scraping data
    2. Data cleaning
  6. Export Filtered Data
    1. File export (CSV, Excel)
    2. Export with a CMS
    3. Export to remote server
  7. Programming with Grimport Script Editor
    1. Introducing Grimport Script Editor
    2. Using Grimport Script Editor
  8. Crawler and Logs
  9. GRS, GRC & GRL
  10. Folder Organization
  11. Launch Options
  12. Cache
  13. Mistakes & Errors
    1. Finding the cause of an error
    2. Common mistakes/errors at the beginning
    3. Error codes (500, 503, etc.)
    4. Java
    5. Other
  14. Connected services
    1. The Cloud
    2. Cloud Endpoint
    3. Third-party APIs with Grimport
  15. To Go Further
    1. Parallelism
    2. Graphical elements in the Wizard
    3. Classes / Objects
    4. Java & Groovy inheritance
    5. Import a script
    6. Good coding practices

Crawler and logs

When using the Crawler, you will be met with a window containing multiple buttons you can click on:


  • This button allows you to send the log report to idIA Tech.


  • Rapports gives you two options:

    • The first one is to write in a file in the log folder the list of visited links.
    • The second ond is to write in a file in the log folder the list of links to visit.


  • This opens the log folder for the current crawl. For more information about the log folder, you may take a look at the Folder Organization page.

  • The current state button has three options.
    • Crawling report which adds a line to the console with information such as the number of commands to execute, the amount of commands left to execute, the number of links left to visited and the amount of links that have been visited. (F2)
    • Complete memory report shows the memory used during the crawl.
    • Limiting factors report shows the amount of time taken by the crawl.
  • (F3)
    Remove any links to visit allows you to choose links to completely avoid by using a regular expression. There are four options:
    • Include, this allows you to make sure specific links are included during the crawl.
    • Exclude once allows you to exclude a link matching the regular expression once.
    • Exclude and keep for this crawl allows you to exclude all links matching the regular expression during the crawl.
    • Exclude and keep forever allows you to exclude all links matching the regular expression for the current crawl and all future crawls.
  • (F12)
    Schedule standby at the end of analysis puts the computer on standby once the crawl has ended. Pressing on it again before the crawl ends will cancel the standby.
  • (F10)
    Display sent commands will display all the commands and functions as they're being executed.

  • The magnifying glass has two options:
    • Display URLs crawled which displays all the URLs that have already had the script run on them. (F7)
    • Display links in the queue which displays all the upcoming URLs that will have the script run on them. (F6)

  • The access button has two options:
    • Protection htaccess/htpasswd, which allows you to enter the login and password of your .htaccess and .htpasswd files if your site has them.
    • Proxy, which allows you to enter the proxy you want to use.
  • (F4)
    This button saves the progress of the crawler to allow you to continue it after interruption.

  • This button opens the Grimport Script Launcher.

  • This button opens Grimport Developer.

  • Change the language to French/English after restarting the software. The flag used matches the language currently used.

  • Update the crawler allows you to update the crawler if it isn't up-to-date.

  • This button takes you to idIA Tech's contact page.

 

Next ❯ ❮ Previous