[es] - Skidanje podataka sa interneta pomocu PHP

nepoverljiv
student
Kragujevac, Srbija

Član broj: 226360
Poruke: 131
*.dynamic.sbb.rs.

+2 Profil

Re: Skidanje podataka sa interneta pomocu PHP

^{06.08.2011. u 18:07 - pre 154 meseci}

Aj da ne otvaram novu temu za stari problem :) Nasao sam neku skriptu za skidanje podataka sa IMDB-a, i poceo da je prilagodjavam sebi. E sad su tu nastali problemi. Kad skinem podatke, nije mi problem da napravim varijable od onih podataka koje nisu array, ali kad treba da uzmem podatke iz array-a ne mogu da se snadjem. Naime, kod mi glumce skine kao array i poredja ih jednog ispod drugog. E sad ja hocu da uzmem svakog glumca (ime i prezime) kao posebnu varijablu i da ga ubacim u bazu. Kod za prikazivanje podataka mi izgleda ovako:

Code:

<?php
include("imdb.php");

$imdb = new Imdb();
$movieArray = $imdb->getMovieInfo("The Godfather"); /* ovako mi lista sve sto dobije od podatala */
echo '<table cellpadding="3" cellspacing="2" border="1" width="80%" align="center">';
foreach ($movieArray as $key=>$value){
    $value = is_array($value)?implode("<br />", $value):$value; /
    echo '<tr>';
    echo '<th align="left" valign="top">' . strtoupper($key) . '</th><td>' . $value . '</td>';
    echo '</tr>';
}
echo '</table>';

echo $movieArray['title'] ;  /* a ovako uzimam pojedinacan podatak, ako nije array */

Kod za prikupljanje podataka mi izgleda ovako:

Code:

<?php
class Imdb
{
    function getMovieInfo($title)
    {
        $arr = array();
        $imdbUrl = 'http://www.imdb.com/title/tt0387199/';
        if($imdbUrl === NULL){
            $arr['error'] = "No Title found in Search Results!";
            return $arr;
        }
        $html = $this->geturl($imdbUrl);
        if(stripos($html, "<meta name=\"application-name\" content=\"IMDb\" />") !== false){
            $arr = $this->scrapMovieInfo($html);
            $arr['imdb_url'] = $imdbUrl;
        } else {
            $arr['error'] = "No Title found on IMDb!";
        }
        return $arr;
    }


    // Scan movie meta data from IMDb page
    function scrapMovieInfo($html)
    {
        $arr = array();
        $arr['title_id'] = $this->match('/<link rel="canonical" href="http:\/\/www.imdb.com\/title\/(tt[0-9]+)\/" \/>/ms', $html, 1);
        $arr['title'] = trim($this->match('/<title>(IMDb \- )*(.*?) $.*?<\/title>/ms', $html, 2));
        $arr['year'] = trim($this->match('/<title>.*?\(.*?([0-9][0-9][0-9][0-9]).*?$.*?<\/title>/ms', $html, 1));
        $arr['rating'] = $this->match('/>([0-9].[0-9])<\/b><span class="mellow">\/10/ms', $html, 1);
        $arr['genres'] = array();
        foreach($this->match_all('/<a.*?>(.*?)<\/a>/ms', $this->match('/Genre.?:(.*?)(<\/div>|See more)/ms', $html, 1), 1) as $m)
        {
           array_push($arr['genres'], $m);
        }

        $arr['creator'] = array();
        foreach($this->match_all('/<a.*?>(.*?)<\/a>/ms', $this->match('/Creator.?:(.*?)(<\/div>|>.?and )/ms', $html, 1), 1) as $m)
        {
            array_push($arr['creator'], $m);
        }
        $arr['cast'] = array();
        foreach($this->match_all('/<td class="name">(.*?)<\/td>/ms', $html, 1) as $m)
        {
            array_push($arr['cast'], trim(strip_tags($m)));
        }
        $arr['karakteri'] = array();
        foreach($this->match_all('/<td class="character">(.*?)<\/td>/ms', $html, 1) as $m)
        {
            array_push($arr['karakteri'], trim(strip_tags($m)));
        }

        //Get extra inforation on  Release Dates and AKA Titles

        $arr['runtime'] = trim($this->match('/Runtime:<\/h4>.*?([0-9]+) min.*?<\/div>/ms', $html, 1));
        if($arr['runtime'] == '') $arr['runtime'] = trim($this->match('/infobar.*?([0-9]+) min.*?<\/div>/ms', $html, 1));
        $arr['awards'] = trim($this->match('/([0-9]+) wins/ms',$html, 1));
        $arr['nominations'] = trim($this->match('/([0-9]+) nominations/ms',$html, 1));
        $arr['storyline'] = trim(strip_tags($this->match('/Storyline<\/h2>(.*?)(<em|<\/p>|<span)/ms', $html, 1)));

        return $arr;
    }
    // ************************[ Extra Functions ]******************************
    function geturl($url)
    {
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
        curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 5.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1");
        $html = curl_exec($ch);
        curl_close($ch);
        return $html;
    }

    function match_all($regex, $str, $i = 0)
    {
        if(preg_match_all($regex, $str, $matches) === false)
            return false;
        else
            return $matches[$i];
    }

    function match($regex, $str, $i = 0)
    {
        if(preg_match($regex, $str, $match) == 1)
            return $match[$i];
        else
            return false;
    }
}
?>

E kako da uzmem podatke posebno tamo gde su array (primera radi kod :
$arr['genres'] = array();
foreach($this->match_all('/<a.*?>(.*?)<\/a>/ms', $this->match('/Genre.?:(.*?)(<\/div>|See more)/ms', $html, 1), 1) as $m)
{
array_push($arr['genres'], $m);
}). Sa ovim kodom on mi izbaci Comedy, Drama, a ja hocu ta dva da razdvojim i posebno da ih ubacim u dva razlicita polja u bazu.

Jel moze neko da pomogne?

Odgovor na temu

nepoverljiv
student
Kragujevac, Srbija

Član broj: 226360
Poruke: 131
*.dynamic.sbb.rs.

+2 Profil

Re: Skidanje podataka sa interneta pomocu PHP

^{09.08.2011. u 14:30 - pre 154 meseci}

Ovako sam ja to zamislio. Prvo imam formu u koju unosim url serije ili filma. Kad kliknem na submit otvara mi novu stranicu na kojoj je isto forma, ali je popunjena sa podacima koje sam skinuo sa IMDB, i sa jos nekim poljima koje rucno popunjavam (da li sam gledao film/seriju, moja ocena, itd..). Mogu u toj formi da napravim da sam rucno ubacujem slike, ali se sve nadam kad bi mogao i to da namestim automatski. Evo ti kod kako izgleda:

Stranica za ubacivanje URL serije ili filma :

Code:

<form name="url" action="index.php?=serije" method="post">
<table>
<tr>
<td>Unesite URL:</td>
<td><input type="text" name="naziv" size="70"></td>
</tr>
<tr><td></td><td><input type="submit" value="Submit" /></td></tr>
</table>
</form>

Sa prethodne me salje na sledecu stranicu koja ima ovakav kod:

Code:

<?php
$imdbUrl = $_POST['naziv'];

include("imdb.php");

$imdb = new Imdb();
$movieArray = $imdb->getMovieInfo("");

echo $success;

?>

               <form name="forma" enctype="multipart/form-data" action="addserije.php method="post">
<table>
<tr>
        <td>IMDB:</td>
        <td><input type="text" name="imdb_id" size="50" value="<?php echo $movieArray['title_id']; ?>"></td>
</tr>
<tr>
        <td>Ime serije:</td>
        <td><input type="text" name="name" size="50" value="<?php echo $movieArray['title'] ;  ?>"></td>
</tr>
<tr>
        <td>Godina:</td>
        <td><input type="text" name="year" size="50" value="<?php echo $movieArray['year'] ; ?>"></td>
</tr>
<tr>
        <td>IMDB rating:</td>
        <td><input type="text" name="rating" size="50" value="<?php echo $movieArray['rating'] ; ?>"></td>
</tr>
<tr>
        <td>IMDB glasova:</td>
        <td><input type="text" name="votes" size="50" value="<?php echo $votes ?>"></td>
</tr>
<tr>
        <td>Creator:</td>
        <td><input type="text" name="creator1" size="50" value="<?php print $movieArray['creator'][0];  ?>"></td>
</tr>
<tr>
        <td>Creator:</td>
        <td><input type="text" name="creator2" size="50" value="<?php print $movieArray['creator'][1]; ?>"></td>
</tr>
<tr>
        <td>Creator:</td>
        <td><input type="text" name="creator3" size="50" value="<?php print $movieArray['creator'][2]; ?>"></td>
</tr>
<tr>
        <td>Genres:</td>
        <td><input type="text" name="genres1" size="5" value="<?php print $movieArray['genres'][0]; ?>"> GB</td>
</tr>
<tr>
        <td>Genres:</td>
        <td><input type="text" name="genres2" size="5" value="<?php print $movieArray['genres'][1]; ?>"> GB</td>
</tr>
<tr>
        <td>Genres:</td>
        <td><input type="text" name="genres3" size="5" value="<?php print $movieArray['genres'][2]; ?>"> GB</td>
</tr>
<tr>
        <td>Genres:</td>
        <td><input type="text" name="genres4" size="5" value="<?php print $movieArray['genres'][3]; ?>"> GB</td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast1" size="50" value="<?php print $movieArray['cast'][0]; ?>"></td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast2" size="50" value="<?php print $movieArray['cast'][1]; ?>"></td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast3" size="50" value="<?php print $movieArray['cast'][2]; ?>"></td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast4" size="50" value="<?php print $movieArray['cast'][3]; ?>"></td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast5" size="50" value="<?php print $movieArray['cast'][4]; ?>"></td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast6" size="50" value="<?php print $movieArray['cast'][5]; ?>"></td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast7" size="50" value="<?php print $movieArray['cast'][6]; ?>"></td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast8" size="50" value="<?php print $movieArray['cast'][7]; ?>"></td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast9" size="50" value="<?php print $movieArray['cast'][8]; ?>"></td>
</tr>
<tr>
        <td>Glumac:</td>
        <td><input type="text" name="cast10" size="50" value="<?php print $movieArray['cast'][9]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast1" size="50" value="<?php print $movieArray['karakteri'][0]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast2" size="50" value="<?php print $movieArray['karakteri'][1]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast3" size="50" value="<?php print $movieArray['karakteri'][2]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast4" size="50" value="<?php print $movieArray['karakteri'][3]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast5" size="50" value="<?php print $movieArray['karakteri'][4]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast6" size="50" value="<?php print $movieArray['karakteri'][5]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast7" size="50" value="<?php print $movieArray['karakteri'][6]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast8" size="50" value="<?php print $movieArray['karakteri'][7]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast9" size="50" value="<?php print $movieArray['karakteri'][8]; ?>"></td>
</tr>
<tr>
        <td>Characters:</td>
        <td><input type="text" name="cast10" size="50" value="<?php print $movieArray['karakteri'][9]; ?>"></td>
</tr>
<tr>
        <td>Vreme trajanja:</td>
        <td><input type="text" name="time" size="50" value="<?php echo $movieArray['runtime'] ; ?>"></td>
</tr>
<tr>
        <td>Osvojene nagrade:</td>
        <td><input type="text" name="wins" size="50" value="<?php echo $movieArray['awards'] ; ?>"></td>
</tr>
<tr>
        <td>Nominacije:</td>
        <td><input type="text" name="nominations" size="50" value="<?php echo $movieArray['nominations'] ; ?>"></td>
</tr>
<tr>
        <td>Storyline:</td>
        <td><textarea name="story" rows=16 cols="60"><?php echo $movieArray['storyline'] ; ?></textarea></td>
</tr>
<tr>
        <td>URL:</td>
        <td><input type="text" name="url" size="50" value="<?php echo $imdbUrl; ?>"></td>
</tr>
<tr>
        <td>Gledao:</td>
            <td><input type="radio" checked="checked" name="gledao" value="0" /> Ne <input type="radio" name="gledao" value="1" /> Da</td></tr>
<tr>
        <td>Ocena:</td>
        <td><input type="text" name="rejting" size="50"></td>
</tr>

<tr></tr>
<tr><td></td><td align="left"><input type="submit" value="Submit"></td></tr>
</table><br>
</form>

I na kraju kod koji mi skida podatke:

Code:

<?php
/////////////////////////////////////////////////////////////////////////////////////////////////////////
// Free PHP IMDb Scraper API for the new IMDb Template.
// Version: 2.3
// Author: Abhinay Rathore
// Website: http://www.AbhinayRathore.com
// Blog: http://web3o.blogspot.com
// Demo: http://lab.abhinayrathore.com/imdb/
// More Info: http://web3o.blogspot.com/2010...per-for-new-imdb-template.html
// Last Updated: Aug 4, 2011
/////////////////////////////////////////////////////////////////////////////////////////////////////////
class Imdb
{
    function getMovieInfo($title)
    {
        $arr = array();
        $imdbUrl = $_POST['naziv'];
        if($imdbUrl === NULL){
            $arr['error'] = "No Title found in Search Results!";
            return $arr;
        }
        $html = $this->geturl($imdbUrl);
        if(stripos($html, "<meta name=\"application-name\" content=\"IMDb\" />") !== false){
            $arr = $this->scrapMovieInfo($html);
            $arr['imdb_url'] = $imdbUrl;
        } else {
            $arr['error'] = "No Title found on IMDb!";
        }
        return $arr;
    }


    // Scan movie meta data from IMDb page
    function scrapMovieInfo($html)
    {
        $arr = array();
        $arr['title_id'] = $this->match('/<link rel="canonical" href="http:\/\/www.imdb.com\/title\/(tt[0-9]+)\/" \/>/ms', $html, 1);
        $arr['title'] = trim($this->match('/<title>(IMDb \- )*(.*?) $.*?<\/title>/ms', $html, 2));
        $arr['year'] = trim($this->match('/<title>.*?\(.*?([0-9][0-9][0-9][0-9]).*?$.*?<\/title>/ms', $html, 1));
        $arr['rating'] = $this->match('/>([0-9].[0-9])<\/b><span class="mellow">\/10/ms', $html, 1);
        $arr['genres'] = array();
        foreach($this->match_all('/<a.*?>(.*?)<\/a>/ms', $this->match('/Genre.?:(.*?)(<\/div>|See more)/ms', $html, 1), 1) as $m)
        {
           array_push($arr['genres'], $m);
        }

        $arr['creator'] = array();
        foreach($this->match_all('/<a.*?>(.*?)<\/a>/ms', $this->match('/Creator.?:(.*?)(<\/div>|>.?and )/ms', $html, 1), 1) as $m)
        {
            array_push($arr['creator'], $m);
        }
        $arr['cast'] = array();
        foreach($this->match_all('/<td class="name">(.*?)<\/td>/ms', $html, 1) as $m)
        {
            array_push($arr['cast'], trim(strip_tags($m)));
        }
        $arr['karakteri'] = array();
        foreach($this->match_all('/<td class="character">(.*?)<\/td>/ms', $html, 1) as $m)
        {
            array_push($arr['karakteri'], trim(strip_tags($m)));
        }

        //Get extra inforation on  Release Dates and AKA Titles

        $arr['runtime'] = trim($this->match('/Runtime:<\/h4>.*?([0-9]+) min.*?<\/div>/ms', $html, 1));
        if($arr['runtime'] == '') $arr['runtime'] = trim($this->match('/infobar.*?([0-9]+) min.*?<\/div>/ms', $html, 1));
        $arr['awards'] = trim($this->match('/([0-9]+) wins/ms',$html, 1));
        $arr['nominations'] = trim($this->match('/([0-9]+) nominations/ms',$html, 1));
        $arr['storyline'] = trim(strip_tags($this->match('/Storyline<\/h2>(.*?)(<em|<\/p>|<span)/ms', $html, 1)));

        return $arr;
    }




    // ************************[ Extra Functions ]******************************
    function geturl($url)
    {
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
        curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 5.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1");
        $html = curl_exec($ch);
        curl_close($ch);
        return $html;
    }

    function match_all($regex, $str, $i = 0)
    {
        if(preg_match_all($regex, $str, $matches) === false)
            return false;
        else
            return $matches[$i];
    }

    function match($regex, $str, $i = 0)
    {
        if(preg_match($regex, $str, $match) == 1)
            return $match[$i];
        else
            return false;
    }

}

E sad bi jos da skinem sliku serije ili filma koja stoji na imdb-u (mada mi za nju nije mnogo ni frka, to bi mogao i rucno da ubacujem), i najvise bi voleo kad bi mogao sa skinem one slicice glumaca i da ih smestim kod mene na racunar u odredjeni folder. Sta mislis, da li je to komplikovano odraditi?

Odgovor na temu