time()，以浮点形式返回自Linux新世纪以来经过的秒数。在linux中，00:00:00 UTC, January 1, 1970是新**49**的开始。

1.open

使用open打开文件后一定要记得调用文件对象的close()方法。比如可以用try/finally语句来确保最后能关闭文件。

file_object = open('thefile.txt')
try:
     all_the_text = file_object.read( )
finally:
     file_object.close( )

注：不能把open语句放在try块里，因为当打开文件出现异常时，文件对象file_object无法执行close()方法。

2.读文件

读文本文件

input = open('data', 'r')
#第二个参数默认为r
input = open('data')

读二进制文件

input = open('data', 'rb')

读取所有内容

file_object = open('thefile.txt')
try:
     all_the_text = file_object.read( )
finally:
     file_object.close( )

读固定字节

file_object = open('abinfile', 'rb')
try:
    while True:
         chunk = file_object.read(100)
        if not chunk:
            break
         do_something_with(chunk)
finally:
     file_object.close( )

读每行

list_of_all_the_lines = file_object.readlines( )

如果文件是文本文件，还可以直接遍历文件对象获取每行：

for line in file_object:
     process line

3.写文件

写文本文件

output = open('data', 'w')

写二进制文件

output = open('data', 'wb')

追加写文件

output = open('data', 'w+')

写数据

file_object = open('thefile.txt', 'w')
file_object.write(all_the_text)
file_object.close( )

写入多行

file_object.writelines(list_of_text_strings)

注意，调用writelines写入多行在性能上会比使用write一次性写入要高。

在处理日志文件的时候，常常会遇到这样的情况：日志文件巨大，不可能一次性把整个文件读入到内存中进行处理，例如需要在一台物理内存为 2GB 的机器上处理一个 2GB 的日志文件，我们可能希望每次只处理其中 200MB 的内容。
在 Python 中，内置的 File 对象直接提供了一个 readlines(sizehint) 函数来完成这样的事情。以下面的代码为例：

file = open('test.log', 'r')

sizehint = 209715200   # 200M

position = 0

lines = file.readlines(sizehint)

while not file.tell() - position < 0:

       position = file.tell()

       lines = file.readlines(sizehint)

http://www.kitebird.com/articles/pydbapi.html

http://www.codegood.com/  python mysql驱动下载大全


the Python standard for database interfaces is the Python DB-API. Most Python database interfaces adhere to this standard.

You can choose the right database for your application. Python Database API supports a wide range of database servers:

• GadFly
• mSQL
• MySQL
• PostgreSQL
• Microsoft SQL Server 2000
• Informix
• Interbase
• Oracle
• Sybase

Here is the list of available Python databases interfaces:

Python Database Interfaces and APIs
            

You must download a separate DB API module for each database you need to access. For example, if you need to access an Oracle database as well as a MySQL database, you must download both the Oracle and the MySQL database modules.

The DB API provides a minimal standard for working with databases, using Python structures and syntax wherever possible. This API includes the following:

• Importing the api module.

• Acquiring a connection with the database.

• Issuing SQL statements and stored procedures.

• Closing the connection

We would learn all the concepts using MySQL so let's talk about MySQLdb module only.

What is MySQLdb?

MySQLdb is an interface for connecting to a MySQL database server from Python. It implements the Python Database API v2.0, and is built on top of the MySQL C API.

How do I install the MySQLdb?

Before proceeding you make sure you have MySQLdb installed on your machine. Just type the following in your Python script and execute it:

#!/usr/bin/python
            import MySQLdb
            

If it produces following result then it means MySQLdb module is not installed:

Traceback (most recent call last):
            File "test.py", line 3, in <module>
            import MySQLdb
            ImportError: No module named MySQLdb
            

To install MySQLdb module, download it from MySQLdb Download page and proceed as follows:

$ gunzip MySQL-python-1.2.2.tar.gz
            $ tar -xvf MySQL-python-1.2.2.tar
            $ cd MySQL-python-1.2.2
            $ python setup.py build
            $ python setup.py install
            

Note: Make sure you have root privilege to install above module.

Database Connection:

Before connecting to a MySQL database make sure followings:

• You have created a database TESTDB.

• You have created a table EMPLOYEE in TESTDB.

• This table is having fields FIRST_NAME, LAST_NAME, AGE, SEX and INCOME.

• User ID "testuser" and password "test123" are set to access TESTDB

• Python module MySQLdb is installed properly on your machine.

• You have gone through MySQL tutorial to understand MySQL Basics.

Example:

Following is the example of connecting with MySQL database "TESTDB"

#!/usr/bin/python
            import MySQLdb
            # Open database connection
            db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
            # prepare a cursor object using cursor() method
            cursor = db.cursor()
            # execute SQL query using execute() method.
            cursor.execute("SELECT VERSION()")
            # Fetch a single row using fetchone() method.
            data = cursor.fetchone()
            print "Database version : %s " % data
            # disconnect from server
            db.close()
            

While running this script, its producing following result at my Linux machine.

Database version : 5.0.45
            

If a connection is established with the datasource then a Connection Object is returned and saved into db for further use otherwise db is set to None. Next db object is used to create a cursor object which in turn is used to execute SQL queries.

Finally before coming out it ensures that database connection is closed and resources are released.

Creating Database Table:

Once a database connection is established, we are ready to create tables or records into the database tables using execute method of the created cursor.

Example:

First let's create Database table EMPLOYEE:

#!/usr/bin/python
            import MySQLdb
            # Open database connection
            db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
            # prepare a cursor object using cursor() method
            cursor = db.cursor()
            # Drop table if it already exist using execute() method.
            cursor.execute("DROP TABLE IF EXISTS EMPLOYEE")
            # Create table as per requirement
            sql = """CREATE TABLE EMPLOYEE (
            FIRST_NAME  CHAR(20) NOT NULL,
            LAST_NAME  CHAR(20),
            AGE INT,
            SEX CHAR(1),
            INCOME FLOAT )"""
            cursor.execute(sql)
            # disconnect from server
            db.close()
            

INSERT Operation:

INSERT operation is required when you want to create your records into a database table.

Example:

Following is the example which executes SQL INSERT statement to create a record into EMPLOYEE table.

#!/usr/bin/python
            import MySQLdb
            # Open database connection
            db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
            # prepare a cursor object using cursor() method
            cursor = db.cursor()
            # Prepare SQL query to INSERT a record into the database.
            sql = """INSERT INTO EMPLOYEE(FIRST_NAME,
            LAST_NAME, AGE, SEX, INCOME)
            VALUES ('Mac', 'Mohan', 20, 'M', 2000)"""
            try:
            # Execute the SQL command
            cursor.execute(sql)
            # Commit your changes in the database
            db.commit()
            except:
            # Rollback in case there is any error
            db.rollback()
            # disconnect from server
            db.close()
            

Above example can be written as follows to create SQL queries dynamically:

#!/usr/bin/python
            import MySQLdb
            # Open database connection
            db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
            # prepare a cursor object using cursor() method
            cursor = db.cursor()
            # Prepare SQL query to INSERT a record into the database.
            sql = "INSERT INTO EMPLOYEE(FIRST_NAME, \
            LAST_NAME, AGE, SEX, INCOME) \
            VALUES ('%s', '%s', '%d', '%c', '%d' )" % \
            ('Mac', 'Mohan', 20, 'M', 2000)
            try:
            # Execute the SQL command
            cursor.execute(sql)
            # Commit your changes in the database
            db.commit()
            except:
            # Rollback in case there is any error
            db.rollback()
            # disconnect from server
            db.close()
            

Example:

Following code segment is another form of execute where you can pass parameters directly:

..................................
            user_id = "test123"
            password = "password"
            con.execute('insert into Login values("%s", "%s")' % \
            (user_id, password))
            ..................................
            

READ Operation:

READ Operation on any databasse means to fetch some useful information from the database.

Once our database connection is established, we are ready to make a query into this database. We can use either fetchone() method to fetch single record or fetchall method to fetech multiple values from a database table.

• fetchone(): This method fetches the next row of a query result set. A result set is an object that is returned when a cursor object is used to query a table.

• fetchall(): This method fetches all the rows in a result set. If some rows have already been extracted from the result set, the fetchall() method retrieves the remaining rows from the result set.

• rowcount: This is a read-only attribute and returns the number of rows that were affected by an execute() method.

Example:

Following is the procedure to query all the records from EMPLOYEE table having salary more than 1000.

#!/usr/bin/python
            import MySQLdb
            # Open database connection
            db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
            # prepare a cursor object using cursor() method
            cursor = db.cursor()
            # Prepare SQL query to INSERT a record into the database.
            sql = "SELECT * FROM EMPLOYEE \
            WHERE INCOME > '%d'" % (1000)
            try:
            # Execute the SQL command
            cursor.execute(sql)
            # Fetch all the rows in a list of lists.
            results = cursor.fetchall()
            for row in results:
            fname = row[0]
            lname = row[1]
            age = row[2]
            sex = row[3]
            income = row[4]
            # Now print fetched result
            print "fname=%s,lname=%s,age=%d,sex=%s,income=%d" % \
            (fname, lname, age, sex, income )
            except:
            print "Error: unable to fecth data"
            # disconnect from server
            db.close()
            

This will produce following result:

fname=Mac, lname=Mohan, age=20, sex=M, income=2000
            

Update Operation:

UPDATE Operation on any databasse means to update one or more records which are already available in the database. Following is the procedure to update all the records having SEX as 'M'. Here we will increase AGE of all the males by one year.

Example:

#!/usr/bin/python
            import MySQLdb
            # Open database connection
            db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
            # prepare a cursor object using cursor() method
            cursor = db.cursor()
            # Prepare SQL query to UPDATE required records
            sql = "UPDATE EMPLOYEE SET AGE = AGE + 1
            WHERE SEX = '%c'" % ('M')
            try:
            # Execute the SQL command
            cursor.execute(sql)
            # Commit your changes in the database
            db.commit()
            except:
            # Rollback in case there is any error
            db.rollback()
            # disconnect from server
            db.close()
            

DELETE Operation:

DELETE operation is required when you want to delete some records from your database. Following is the procedure to delete all the records from EMPLOYEE where AGE is more than 20.

Example:

#!/usr/bin/python
            import MySQLdb
            # Open database connection
            db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
            # prepare a cursor object using cursor() method
            cursor = db.cursor()
            # Prepare SQL query to DELETE required records
            sql = "DELETE FROM EMPLOYEE WHERE AGE > '%d'" % (20)
            try:
            # Execute the SQL command
            cursor.execute(sql)
            # Commit your changes in the database
            db.commit()
            except:
            # Rollback in case there is any error
            db.rollback()
            # disconnect from server
            db.close()
            

Performing Transactions:

Transactions are a mechanism that ensures data consistency. Transactions should have the following four properties:

• Atomicity: Either a transaction completes or nothing happens at all.

• Consistency: A transaction must start in a consistent state and leave the system is a consistent state.

• Isolation: Intermediate results of a transaction are not visible outside the current transaction.

• Durability: Once a transaction was committed, the effects are persistent, even after a system failure.

The Python DB API 2.0 provides two methods to either commit or rollback a transaction.

Example:

You already have seen how we have implemented transations. Here is again similar example:

# Prepare SQL query to DELETE required records
            sql = "DELETE FROM EMPLOYEE WHERE AGE > '%d'" % (20)
            try:
            # Execute the SQL command
            cursor.execute(sql)
            # Commit your changes in the database
            db.commit()
            except:
            # Rollback in case there is any error
            db.rollback()
            

COMMIT Operation:

Commit is the operation which gives a green signal to database to finalize the changes and after this operation no change can be reverted back.

Here is a simple example to call commit method.

    db.commit()
            

ROLLBACK Operation:

If you are not satisfied with one or more of the changes and you want to revert back those changes completely then use rollback method.

Here is a simple example to call rollback metho.

   db.rollback()
            

Disconnecting Database:

To disconnect Database connection, use close() method.

    db.close()
            

If the connection to a database is closed by the user with the close() method, any outstanding transactions are rolled back by the DB. However, instead of depending on any of DB lower level implementation details, your application would be better off calling commit or rollback explicitly.

Handling Errors:

There are many sources of errors. A few examples are a syntax error in an executed SQL statement, a connection failure, or calling the fetch method for an already canceled or finished statement handle.

The DB API defines a number of errors that must exist in each database module. The following table lists these exceptions.

Your Python scripts should handle these errors but before using any of the above exceptions, make sure your MySQLdb has support for that exception. You can get more information about them by reading the DB API 2.0 specification.

import urllib2

socket.setdefaulttimeout(seconds)

open = urllib2.urlopen("http://www.shuoqian.net")

过socket.setdefaulttimeout() 设置了全局默认超时时间，从而给urllibe2.urlopen()也设置了默认的超时时间





抓取图片(这个例子里的Request找不到，我一般只有urllib2)
soup=urlllib2.open(url)

# Let's create a function that downloads a file, and saves it locally.
# This function accepts a file name, a read/write mode(binary or text),
# and the base url.

def stealStuff(file_name,file_mode,base_url):
    from urllib2 import Request, urlopen, URLError, HTTPError

    #create the url and the request
    url = base_url + file_name
    req = Request(url)

    # Open the url
    try:
        f = urlopen(req)
        print "downloading " + url

        # Open our local file for writing
        local_file = open(file_name, "w" + file_mode)
        #Write to our local file
        local_file.write(f.read())
        local_file.close()

    #handle errors
    except HTTPError, e:
        print "HTTP Error:",e.code , url
    except URLError, e:
        print "URL Error:",e.reason , url

# Set the range of images to 1-50.It says 51 because the
# range function never gets to the endpoint.
image_range = range(1,51)

# Iterate over image range
for index in image_range:

    base_url = 'http://www.techniqal.com/'
    #create file name based on known pattern
    file_name =  str(index) + ".jpg"
    # Now download the image. If these were text files,
    # or other ascii types, just pass an empty string
    # for the second param ala stealStuff(file_name,'',base_url)
    stealStuff(file_name,"b",base_url)

字符串中字符大小写的变换：

    * S.lower()   #小写
    * S.upper()   #大写
    * S.swapcase()   #大小写互换
    * S.capitalize()   #首字母大写
    * String.capwords(S) 
      #这是模块中的方法。它把S用split()函数分开，然后用capitalize()把首字母变成大写，最后用join()合并到一起
    * S.title()    #只有首字母大写，其余为小写，模块中没有这个方法

字符串在输出时的对齐：

    * S.ljust(width,[fillchar]) 
      #输出width个字符，S左对齐，不足部分用fillchar填充，默认的为空格。
    * S.rjust(width,[fillchar])    #右对齐
    * S.center(width, [fillchar])    #中间对齐
    * S.zfill(width)   #把S变成width长，并在右对齐，不足部分用0补足

字符串中的搜索和替换：

    * S.find(substr, [start, [end]]) 
      #返回S中出现substr的第一个字母的标号，如果S中没有substr则返回-1。start和end作用就相当于在S[start:end]中搜索
    * S.index(substr, [start, [end]]) 
      #与find()相同，只是在S中没有substr时，会返回一个运行时错误
    * S.rfind(substr, [start, [end]]) 
      #返回S中最后出现的substr的第一个字母的标号，如果S中没有substr则返回-1，也就是说从右边算起的第一次出现的substr的首字母标号
    * S.rindex(substr, [start, [end]])
    * S.count(substr, [start, [end]])    #计算substr在S中出现的次数
    * S.replace(oldstr, newstr, [count])  
      #把S中的oldstar替换为newstr，count为替换次数。这是替换的通用形式，还有一些函数进行特殊字符的替换
    * S.strip([chars])
      #把S中前后chars中有的字符全部去掉，可以理解为把S前后chars替换为None
    * S.lstrip([chars])
    * S.rstrip([chars])
    * S.expandtabs([tabsize]) 
      #把S中的tab字符替换没空格，每个tab替换为tabsize个空格，默认是8个

字符串的分割和组合：

    * S.split([sep, [maxsplit]])
      #以sep为分隔符，把S分成一个list。maxsplit表示分割的次数。默认的分割符为空白字符
    * S.rsplit([sep, [maxsplit]])
    * S.splitlines([keepends])
      #把S按照行分割符分为一个list，keepends是一个bool值，如果为真每行后而会保留行分割符。
    * S.join(seq) #把seq代表的序列──字符串序列，用S连接起来

字符串的mapping，这一功能包含两个函数：

    * String.maketrans(from, to)
      #返回一个256个字符组成的翻译表，其中from中的字符被一一对应地转换成to，所以from和to必须是等长的。
    * S.translate(table[,deletechars])
      #使用上面的函数产后的翻译表，把S进行翻译，并把deletechars中有的字符删掉。需要注意的是，如果S为unicode字符串，那么就不支持 deletechars参数，可以使用把某个字符翻译为None的方式实现相同的功能。此外还可以使用codecs模块的功能来创建更加功能强大的翻译表。

字符串还有一对编码和解码的函数：

    * S.encode([encoding,[errors]])
      #其中encoding可以有多种值，比如gb2312 gbk gb18030 bz2 zlib big5 bzse64等都支持。errors默认值为"strict"，意思是UnicodeError。可能的值还有'ignore', 'replace', 'xmlcharrefreplace', 'backslashreplace' 和所有的通过codecs.register_error注册的值。这一部分内容涉及codecs模块，不是特明白
    * S.decode([encoding,[errors]])

字符串的测试函数，这一类函数在string模块中没有，这些函数返回的都是bool值：

    * S.startwith(prefix[,start[,end]])
      #是否以prefix开头
    * S.endwith(suffix[,start[,end]])
      #以suffix结尾
    * S.isalnum()
      #是否全是字母和数字，并至少有一个字符
    * S.isalpha()  #是否全是字母，并至少有一个字符
    * S.isdigit()  #是否全是数字，并至少有一个字符
    * S.isspace() #是否全是空白字符，并至少有一个字符
    * S.islower() #S中的字母是否全是小写
    * S.isupper() #S中的字母是否便是大写
    * S.istitle() #S是否是首字母大写的

字符串类型转换函数，这几个函数只在string模块中有：

    * string.atoi(s[,base])
      #base默认为10，如果为0,那么s就可以是012或0x23这种形式的字符串，如果是16那么s就只能是0x23或0X12这种形式的字符串
    * string.atol(s[,base])  #转成long
    * string.atof(s[,base])  #转成float

*******************************************************************************************************************
Python字符串在输出时的对齐：

    
S.ljust(width,[fillchar])   

#输出width个字符，S左对齐，不足部分用fillchar填充，默认的为空格。

    
S.rjust(width,[fillchar]) #  

右对齐

    
S.center(width, [fillchar]) #  

中间对齐 S.zfill(width) #把S变成width长，并在右对齐，不足部分用0补足字符串中的搜索和替换：

    
S.find(substr, [start, [end]])   

#返回S中出现substr的第一个字母的标号，如果S中没有substr则返回-1。start和end作用就相当于在S[start:end]中搜索

    
S.index(substr, [start, [end]])   

#与find()相同，只是在S中没有substr时，会返回一个运行时错误

    
S.rfind(substr, [start, [end]])   

#返回S中最后出现的substr的第一个字母的标号，如果S中没有substr则返回-1，也就是说从右边算起的第一次出现的substr的首字母标号

    
S.rindex(substr, [start, [end]])   
    
S.count(substr, [start, [end]]) #  

计算substr在S中出现的次数

    
S.replace(oldstr, newstr, [count])   

#把S中的oldstar替换为newstr，count为替换次数。这是替换的通用形式，还有一些函数进行特殊字符的替换

**************************************************************************************************