Waxaa iibiyay Zinn Digital
Qalabka xogta internetka aruuriya ee gaarka ah ee ku dhisan Python ama Node.js loona sameeyay bogagga dhabta ah, kuwaas oo maareeya soo-saarista JS, gaashaanka ka hortagga aaladaha otomaatiga ah iyo baarista bogagga badan, iyadoo natiijo nadiif ah oo CSV, JSON ama xog-ururin ah lagu keenayo 2 ilaa 10 maalmood gudahood.
Aaladaha xog-ururinta ee loo dhisay bogagga dhabta ah, ee aan ahayn kuwa tusaalaha ah
Aaladaha xog-soo-saarka ee u diyaarsan shaqada waxaa lagu dhisay Python ama Node.js looguna talagalay buug-yareyaasha alaabta, tusmooyinka, suuqyada ganacsiga, iyo barnaamijyada firfircoon ee hal-bogga ah. Natiijadu waxay ku imaanaysaa CSV, JSON, ama si toos ah ayaa loogu qorayaa keydkaaga xogta.
Waxyaabaha ay ka kooban tahay
Sababta ay u shaqeyso
Soo-saarayaal badan ayaa hal mar shaqeeya marka la tijaabinayo, waxayna guuldarraystaan isla marka goobtu beddesho calaamadeynta, xannibto user agent, ama ku darto lakabyo ilaalin oo cusub. Dhismooyinku waxay ku tiirsan yihiin doorayaal adkeysi leh, nidaamka dib-u-isku-dayga, kaydinta wakiilka (proxy fallback) iyo diiwaangelin cad, markaa marka doore jabto ama boggu soo celiyo khalad halkii uu xog ka bixin lahaa, waxaad si sax ah u arkaysaa diiwaannada fashilmay iyo sababta, halkii aad ka heli lahayd safaf madhan oo aamusan.
Aaladaha la isticmaalo
Python (BeautifulSoup, Scrapy, Playwright, Selenium, Requests) iyo Node.js (Puppeteer, Cheerio, Playwright, Axios) oo loo isticmaalo soo-saarista; wakiillo wareegaya iyo wareegga user-agent oo lagu daray far-raaca biraawsarka madaxa-la'aan ee hawsha ka-hortagga bot-ka; kaydinta dhammaan CSV, JSON, Excel, Google Sheets, PostgreSQL, MySQL, MongoDB, SQLite iyo Airtable; jadwalka iyada oo loo marayo cron, GitHub Actions, AWS Lambda, Vercel Cron ama Railway; kaabayaasha ku yaal Docker, bixiyeyaasha VPS, AWS ama Render.
Tijaabinta ka hor wareejinta
Dhisid kasta waxay marka hore ku shaqaysaa tijaabo qallalan oo ku saabsan muunad yar, ka dibna orod buuxa oo leh warbixin kooban oo guul iyo guuldarro ah. Wax-soo-saarka waxaa lagu hubiyaa qaab-dhismeedka la filayo, iyadoo meelaha bannaanka ah ee maqan, safafka nuqullada ah iyo khaladaadka kala-saaridda la calaamadiyo ka hor keenista.
Lahaanshaha
Koodhka asalka ah iyo aqoonsiga soo-saaraha si buuxda ayaa lagugu wareejinayaa marka la bixiyo ugu dambaynta. Heshiiska xog-qarinta ayaa la heli karaa marka la codsado mashaariicda xogta xasaasiga ah ama tartanka leh.
Jadwalka waqtiga
Aasaasiga ah (Basic) waxa lagu diraa 2 ilaa 3 maalmood gudahood, Heerka caadiga ah (Standard) 5 ilaa 7 maalmood, iyo Qaybta sare (Premium) 10 maalmood gudahood.
Waxaa lagaa rabaa
Soo dir goobta bartilmaameedka ah iyo liiska qaybaha, waxaana laguugu diyaarin karaa horu-u-arag shaqeynaya oo qalabka xogta soo saara ah muddo 48 saacadood gudahood ah, ka hor inta aan wax lacag bixin ah loo baahan, si aad marka hore u aragto xog dhab ah oo laga soo saaray bartilmaameedkaaga.
$50.99
Ama ku bixi €45.51 · £38.11 qiimaha maanta — qiimaha waa la xirayaa marka aad ku darto dambiishaada.
Single-site, single-page scraper with clean CSV or JSON output. - Full source code + setup - One target URL scraped - Up to 10 fields extracted per item - Static HTML parsing (no JS rendering) - CSV or JSON output - Deduplication + field normalization - Basic error handling and logging - Readme - 2 revisions
La keenay gudaha 3 maalmood
Dib-u-eegisyada ku jira: 0
$125.99
Ama ku bixi €112.45 · £94.17 qiimaha maanta — qiimaha waa la xirayaa marka aad ku darto dambiishaada.
Multi-page crawl with pagination, JS rendering and anti-bot handling. - Full source code + setup - One target URL scraped - Static HTML parsing (no JS rendering) - Deduplication + field normalization - Basic error handling and logging - Readme - Structured error log + retry queue - Multi-page crawl with pagination support - JavaScript-rendered pages (Puppeteer or Playwright) - Anti-bot handling — UA rotation, headers, request throttling - Proxy rotation support - Rate-limit compliance with backoff on 429/503 - Up to 25 fields per item - Unlimited records - Output to CSV, JSON, or Google Sheets - 3 revisions
La keenay gudaha 7 maalmood
Dib-u-eegisyada ku jira: 0
$375.99
Ama ku bixi €335.60 · £281.04 qiimaha maanta — qiimaha waa la xirayaa marka aad ku darto dambiishaada.
Full data pipeline with scheduling, database output, monitoring and dashboard. - Full source code + setup - One target URL scraped - Static HTML parsing (no JS rendering) - Deduplication + field normalization - Basic error handling and logging - Readme - Structured error log + retry queue - Multi-page crawl with pagination support - JavaScript-rendered pages (Puppeteer or Playwright) - Anti-bot handling — UA rotation, headers, request throttling - Proxy rotation support - Rate-limit compliance with backoff on 429/503 - Up to 25 fields per item - Unlimited records - Output to CSV, JSON, or Google Sheets - Priority delivery slot - Unlimited fields + custom schema mapping - Database output — PostgreSQL, MySQL, MongoDB, or SQLite - Scheduled runs — cron, GitHub Actions, AWS Lambda or Vercel - Cron Change detection between runs (new / updated / removed records) - Login-gated content support (session handling, form submission) - CAPTCHA-aware flow (manual-solve integration or third-party solver) - Docker-based deployment - Monitoring dashboard or Slack/email alerts on failure - Loom walkthrough of the full pipeline - 30 days post-delivery support - 5 revisions
La keenay gudaha 10 maalmood
Dib-u-eegisyada ku jira: 0
Marka hore akoon looma baahna — kaaga waxaa la samaynayaa marka la bixinayo lacagta. Lacagta waannu haynaynaa ilaa aad ka aqbasho shaqada.
Xaalado badan, haa. Waxaan isku darnaa wareegga wakiilka (proxy), kala duwanaanta wakiilka isticmaalaha, xakamaynta xawaaraha iyo muujinta biraawsarka aan madaxa lahayn si aan uga gudubno xannibaadaha caadiga ah. Goobaha isticmaala adeegyada ilaalinta ee culus waxay kordhin karaan kharashka, markaa waxaan marka hore dib u eegnaa bartilmaameedka oo aan kuu sheegnaa haddii uu ku habboon yahay qiimaha caadiga ah ama haddii uu u baahan yahay qiimeyn gaar ah.
Soo saarista xogta boggaga si guud loo heli karo guud ahaan waa la oggol yahay meelaha badankood, in kasta oo ay ku xidhan tahay shuruudaha goobta, xogta ku lugta leh iyo sida aad ugu talogashay inaad u isticmaasho. Waxaan la shaqaynaa oo keliya boggaga guud, ma dhaafrno xannibaadaha lacag-bixinta (paywalls), waxaan ka fogaannaa ururinta xogta shakhsiyeed, mana jabin doonno shuruudaha cad ee ka dhanka ah soo saarista xogta annaga oo aan haysan oggolaanshahaaga qoraalka ah iyo caddaynta sharciga.
Qaababka bogagga way isbeddelaan, natiijo ahaanna doorayaashu way joojin karaan shaqada. Xoqdo kasta waxaa ku jira diiwaangelinta khaladaadka habaysan si lagugu ogeysiiyo isla marka goobtu joojiso soo celinta xogta. Standard iyo Premium waxay ku daraan astaamo adkeysi leh sida doorayaasha kaydka ah iyo digniinaha si loo xaddido cilladaha, waxaanan si bilaash ah ku daboolnaa hagaajin kasta oo la xiriirta qaabeynta inta lagu jiro 14-ka maalmood ee ka dambeeya keenista.
Haa, iyadoo la adeegsanayo Puppeteer, Playwright ama Selenium si loo maareeyo barnaamijyada hal-bogga ah, soo-dejinta tartiib-tartiibka ah iyo duudduubka aan dhammaadka lahayn. Tani waa qayb ka mid ah xirmooyinka Standard iyo Premium; Basic waxay ku kooban tahay HTML ma guurto ah.
Haa, xisaabaadka aad leedahay ama aad fasaxa u haysato inaad isticmaasho. Maareynta fadhiga, maamulidda cookies-ka iyo gudbinta foomamka waxay ku jiraan Premium. Ku wadaag aqoonsi kasta waddo ammaan ah; ma sii hayno mashruuca kadib.
Waxaan bixinnaa labo hab: hakad lagu xalliyo gacanta oo aad adigu soo farageliso, ama adeeg xallinta CAPTCHA oo dhinac saddexaad ah oo kharashkiisa aad adigu bixiso. Tani waxay ku dhex dhisan tahay Premium waxaana lagu dari karaa Standard.
Haa, Premium waxay taageertaa socodsiinta la qorsheeyay iyadoo la adeegsanayo cron, GitHub Actions, AWS Lambda ama Vercel Cron. Waxaad dooranaysaa inta jeer ee ay dhacayso, hadday tahay maalinle, toddobaadle, bille ama waqti kale, socodsiin kastaana waxay diiwaangelisaa natiijooyinkeeda iyadoo leh digniino ikhtiyaari ah oo ku saabsan guuldarrada.
Maareynta xaddidka heerka waa mid caadi ah: ceejinta codsiga, dib-u-dhac jibbaaran oo ku saabsan jawaabaha 429 iyo 503, iyo habaynta isku-simanka si loogu jiro xadka dhexdooda. Labada Standard iyo Premium labaduba waxay si buuxda ugu dhex jiraan caqligan.
Haa, gelinta tooska ah ee PostgreSQL, MySQL, MongoDB, SQLite ama Airtable waa la taageeraa. Soosaarista xog-ururintu waa qayb ka mid ah Premium, halka Standard uu daboolayo CSV, JSON iyo Google Sheets.
Ma jiro xad sare oo go'an; waxay ku xiran tahay xaddidaadaha xawaaraha goobta iyo baaxadda kaydka wakiilka (proxy pool) ee la heli karo. Waxaan fulinay shaqooyin soo saaray in ka badan 100,000 oo diiwaan dhowr maalmood gudahood. Xog-ururinta aadka u ballaaran, waxaan soo jeedinaynaa wareejinta wakiilka (proxy rotation) iyo socodsiinta qorsheysan si looga fogaado heerarka ogaanshaha.
Waxaad helaysaa lahaansho buuxda marka ugu dambaysa ee lagu wareejiyo, oo ay ku jiraan koodhka asalka ah, xogta gelitaanka, habaynta hawlgelinta iyo dhammaan xogta la ururiyay, iyada oo aanay jirin wax xaddidaad laysan ah, khidmado joogto ah, ama xannibaad.
Scraper kasta waxaa marka hore lagu tijaabiyaa muunad yar, ka dibna si buuxda ayaa loo hawlgeliyaa. Natiijada waxaa la barbar dhigaa qaabka la filayo, waxaana meelaha banaan ee ka maqan, xariiqyada soo noqnoqda iyo khaladaadka fasiraadda la calaamadeeyaa ka hor wareejinta, iyadoo la raacinayo diiwaanka khaladaadka iyo soo koobidda waxyaabaha guuleystay iyo kuwa fashilmay.