1 download privoxy from website and install it
2 after installation check process
ps aux|grep privoxy
3 if you want change ssh tunnel to http, do modification config file in
/usr/local/etc/privoxy/config
in listen-address
listen-address 127.0.0.1:8118
to
listen-address 0.0.0.0:8118
in forward-socks5
forward-socks5 / 127.0.0.1:9050 .
to your ssh tunnel. eg:
forward-socks5 / 192.168.1.106:7070 .
interface: run()
URL mapping:mapping rule as follows:
normal usage
| path | controller |
|---|---|
| / | packagename.controller.IndexController |
| /xxx | packagename.controller.XxxController |
| /xxx/ | packagename.controller.xxx.IndexController |
| /xxx/yyy | packagename.controller.xxx.YyyController |
| /xxx/yyy/zzz | packagename.controller.xxx.yyy.ZzzController |
note: packagename is defined in web.xml
if you are use GWT add “server” after packagename
url rewrite
Define packagename.controller.AppRouter first.
for example:
package slim3.demo.controller;
import org.slim3.controller.router.RouterImpl;
public class AppRouter extends RouterImpl {
public AppRouter() {
addRouting("/_ah/mail/{address}", "/mail/receive?address={address}");
addRouting(
"/{app}/edit/{key}/{version}",
"/{app}/edit?key={key}&version={version}");
addRouting(
"/{app}/delete/{key}/{version}",
"/{app}/delete?key={key}&version={version}");
addRouting(
"/{app}/find/*path",
"/{app}/find?path={path}");
}
}
The request for /_ah/mail/xxx@gmail.com is routed to /mail/receive?address=xxx@gmail.com.
The request for /blog/edit/xxx/1 is routed to /blog/edit?key=xxx&version=1.
The request for /blog/delete/xxx/1 is routed to /blog/delete?key=xxx&version=1.
The request for /blog/find/xxx/yyy/zzz is routed to /blog/find?path=xxx%2Fyyy%2Fzzz, (xxx/yyy/zzz gets URL-encoded).
A place holder({xxx}) does not match “/”, because “/” is a path separator.
A catch-all place holder(*xxx) must be at the end of the match (From) string and matches everything including “/” up to the end.
file upload
The uploaded result is stored in FileItem.java. So you can get it as follows:
FileItem formFile = requestScope("formFile");
The size of uploaded data is limited to 10MB. The maximum entity size is 1MB. so: UploadedDataFragment:
@Model
public class UploadedData {
@Attribute(primaryKey = true)
private Key key;
@Attribute(version = true)
private Long version = 0L;
private String fileName;
private int length;
@Attribute(persistent = false)
private InverseModelListRef<UploadedDataFragment, UploadedData> fragmentListRef =
new InverseModelListRef<UploadedDataFragment, UploadedData>(
UploadedDataFragment.class,
"uploadDataRef",
this,
new Sort("index"));
...
}
UploadedDataFragment
@Model
public class UploadedDataFragment {
@Attribute(primaryKey = true)
private Key key;
@Attribute(lob = true)
private byte[] bytes;
private ModelRef<UploadedData> uploadDataRef =
new ModelRef<UploadedData>(
UploadedData.class);
private int index;
...
}
UploadService
public UploadedData upload(FileItem formFile) {
if (formFile == null) {
return null;
}
List<Object> models = new ArrayList<Object>();
UploadedData data = new UploadedData();
models.add(data);
data.setKey(Datastore.allocateId(d));
data.setFileName(formFile.getShortFileName());
data.setLength(formFile.getData().length);
byte[] bytes = formFile.getData();
byte[][] bytesArray = ByteUtil.split(bytes, FRAGMENT_SIZE);
Iterator<Key> keys =
Datastore.allocateIds(data.getKey(), f, bytesArray.length).iterator();
for (int i = 0; i < bytesArray.length; i++) {
byte[] fragmentData = bytesArray[i];
UploadedDataFragment fragment = new UploadedDataFragment();
models.add(fragment);
fragment.setKey(keys.next());
fragment.setBytes(fragmentData);
fragment.setIndex(i);
fragment.getUploadDataRef().setModel(data);
}
Transaction tx = Datastore.beginTransaction();
for (Object model : models) {
Datastore.put(tx, model);
}
tx.commit();
return data;
}
validation
Slim3 provides a simple validation framework as follows:
Validators v = new Validators(request);
v.add("arg1", v.required(), v.integerType());
v.add("arg2", v.required(), v.integerType(), v.longRange(3, 5));
if (v.validate()) {
//OK
} else {
//NG
Errors errors = v.getErrors();
System.out.println(errors.get("arg1"));
System.out.println(errors.get("arg2"));
}
The error messages are stored in
validator.required={0} is required.
validator.byteType={0} must be a byte.
validator.shortType={0} must be a short.
validator.integerType={0} must be an integer.
validator.longType={0} must be a long.
validator.floatType={0} must be a float.
validator.doubleType={0} must be a double.
validator.numberType={0} is not a number({1}).
validator.dateType={0} is not a date({1}).
validator.minlength={0} can not be less than {1} characters.
validator.maxlength={0} can not be greater than {1} characters.
validator.range={0} is not in the {1} to {2} range.
validator.regexp={0} is invalid.
{0} is an attribute name. If you want to change the attribute name by locale, define label.
label.arg1=xxx
label.arg2=yyy
If the attribute is stored to a model, you can specify it type-safely using Meta data of model:
Validators v = new Validators(request);
BlogMeta meta = BlogMeta.get();
v.add(meta.title, v.required());
v.add(meta.content, v.required());
To declare a Java class as model, give the class a @Model annotation. For example:
import org.slim3.datastore.Model;
@Model
public class Employee {
// ...
}
Fields of the data class are stored in the datastore. A persistent field must have getter and setter methods.
import java.util.Date;
// ...
private Date hireDate;
public Date getHireDate() {
return hireDate;
}
public void setHireDate(Date hireDate) {
this.hireDate = hireDate;
}
A data class must have one field dedicated to storing the primary key of the corresponding datastore entity. A key field must be com.google.appengine.api.datastore.Key. A key use a @Attribute(primaryKey = true) annotation:
import com.google.appengine.api.datastore.Key;
import org.slim3.datastore.Attribute;
// ...
@Attribute(primaryKey = true)
private Key key;
// ... accessors ...
Serializable Objects
You can store an instance of a Serializable class as a Blob value. To tell Slim3 Datastore to serialize the value, the field uses the annotation @Attribute(lob = true). Blob values are not indexed and cannot be used in query filters or sort orders.
Here is an example of a simple Serializable class that represents a file, including the file contents, a filename and a MIME type.
import java.io.Serializable;
public class DownloadableFile implements Serializable {
private static final long serialVersionUID = 1L;
private byte[] content;
private String filename;
private String mimeType;
// ... accessors ...
}
To store an instance of a Serializable class as a Blob value in a property, declare a field whose type is the class, and use the @Attribute(lob = true) annotation:
import DownloadableFile;
import org.slim3.datastore.Attribute;
// ...
@Attribute(lob = true)
private DownloadableFile file;
Creating a model
To store a model in the datastore, you call Datastore.put() method, passing it the instance.
Employee employee = new Employee();
...
Datastore.put(employee);
Getting a model by Key
Key key = ...;
Employee emp = Datastore.get(Employee.class, key);
Updating a model
To update a model, fetch the model, modify it, and put it to the datastore:
public void updateEmployeeTitle(Key key, String newTitle) {
Employee emp = Datastore.get(Employee.class, key);
emp.setTitle(newTitle);
Datastore.put(emp);
}
Deleting a model
To delete a model from the datastore, call Datastore.delete() method with the key:
Key key = ...;
Datastore.delete(key);
Query
A filter specifies a field name, an operator, and a value. The value must be provided by the app; it cannot refer to another property, or be calculated in terms of other properties. The operator can be any of the following: “equal”, “lessThan”, “lessThanOrEqual”, “greaterThan”, “greaterThanOrEqual”, “isNotNll”, “startsWith”, “notEqual”, “in”. An entity must match all filters to be a result. The logical combination of filters is “and”.
The “notEqual” operator actually performs 2 queries: one where all other filters are the same and the “notEqual” filter is replaced with a “lessThan” filter, and one where the “notEqual” filter is replaced with a “greaterThan” filter. The results are merged, in order. As described below in the discussion of inequality filters, a query can only have one “notEqual” filter, and such a query cannot have other inequality filters.
The “in” operator also performs multiple queries, one for each item in the provided list value where all other filters are the same and the “in” filter is replaced with an equal filter. The results are merged, in the order of the items in the list. If a query has more than 1 “in” filter, the query is performed as multiple queries, one for each combination of values in the “in” filters.
A single query containing “notEqual” or “in” operators is limited to 30 sub-queries.
Date hireDate = ...;
EmployeeMeta e = EmployeeMeta.get();
List<Employee> list = Datastore.query(e)
.filter(e.lastName.equal("Smith"), e.hireDate.greaterThan(hireDate))
.asList();
install Java
install eclipse
install Google Plugin for Eclipse in
http://dl.google.com/eclipse/plugin/x.x
note : “x.x” due to your eclipse version.
install slim3 for Eclipse in
http://slim3.googlecode.com/svn/updates/
set Eclipse for testing
Preference > Java > Code Style > Organize Imports set “1” to “Number of static imports needed for.”
Preference > Java > Editor > Content Assist > Favorites add
org.hamcrest.CoreMatchers
org.junit.Assert
org.junit.matchers.JUnitMatchers
Preference > General > Workspace:
check “Refresh automatically” (eclipse 3.7 above does not have this option)
in eclipse New > project > Slim3 > Silm3 Project > next > give a project name and root package name (should be same)
open build.xml in your project
right click menu Run as > Ant Build (first item)
will popup a dialog,then input “/twitter/” (“twitter/” is short for “/twitter/index”) click “ok” button.
“twitter” is the controller name, so it will automatically create controller class:
projectname.controller.twitter.IndexController
projectname.controller.twitter.IndexControllerTest
war/twitter/index.jsp
Now,you can run testcase for IndexControllerTest by right menu > Run as > JUnit Test. It will show a green progressbar for no problem.
You can run the project by right menu > Run as > Web Application.
Open browser with “http://localhost:8888/twitter/”,and see what happend
Right menu build.xml > Run as > Ant Build (second item ) > gen-model
will popup a dialog to input a model name. let’s name it “Tweet”.
then will automatic add a class Tweet.java
Add field to model In Tweet.java add 2 fields
private String content;
private Date createDate = new Date();
and add the getter and setter for class.
Add service for model
Right click build.xml > Run as > Ant Build(second item) > gen-model > in shown popup dialog, give a name with “TweetService”,then will create an empty service class file named TweetService.java.
Now,change this class:
…
import projectname.model.Tweet;
public class TweeterService{
public Tweet tweet(Map(String,Object) input){
Tweet tweet = new Tweet();
BeanUtil.copy(input,tweet);
Transaction tx = Datastore.beginTransaction();
Datastore.put(tweet);
tx.commit()
return tweet;
}
}
note:
BeanUtil copies property values from the input data to the model for all cases where the property names are the same. Even if the property type of the input is different from the one of the model, the value is converted appropriately. To put a model to the datastore, you use Datastore.put(Object model). See Creating a model. If an exception occurred during the transaction, it is rolled back by Slim3 automatically.
Create getList method
add code to the class:
…
public class TwitterService {
private TweetMeta t = new TweetMeta();
…
public List<Tweet> getTweetList() {
return Datastore.query(t).sort(t.createdDate.desc).asList();
}
}
Go to IndexController.java,change some code to
…
import projectname.model.Tweet;
import projectname.service.TwitterService;
public class IndexController extends Controller {
private TwitterService service = new TwitterService();
@Override
public Navigation run() throws Exception {
List<Tweet> tweetList = service.getTweetList();
requestScope("tweetList", tweetList);
return forward("index.jsp");
}
}
note:requestScope(“tweetList”, tweetList) is short for request.setAttribute(“tweetList”, tweetList)
To render to template,just change the index.jsp file
<%@page pageEncoding="UTF-8" isELIgnored="false"%>
<%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core"%>
<%@taglib prefix="fmt" uri="http://java.sun.com/jsp/jstl/fmt"%>
<%@taglib prefix="fn" uri="http://java.sun.com/jsp/jstl/functions"%>
<%@taglib prefix="f" uri="http://www.slim3.org/functions"%>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>twitter Index</title>
<link rel="stylesheet" type="text/css" href="/css/global.css" />
</head>
<body>
<p>What are you doing?</p>
<c:forEach var="e" items="${tweetList}">
${f:h(e.content)}
<hr />
</c:forEach>
</body>
</html>
note:I list tweeted contents using the tweetList variable. f:h is a JSP function to escape HTML.
Now if you have data in the tweetList,the page will show all the data.content in the list.
reverse:
jQuery.fn.reverse = [].reverse;
object exists:
jQuery.fn.exists = function(){
return this.length>0;
}
string format:
if (!String.prototype.format) {
String.prototype.format = function() {
var args = arguments;
return this.replace(/{(\d+)}/g, function(match, number) {
return typeof args[number] != 'undefined'? args[number]: match;
});
};
}
plugins > fileDownlad.js
settings.py
current directory
import os
APP_ROOT = os.path.dirname(__file__)
models
dynamic file upload path
class THistory(models.Model):
shop = models.ForeignKey('MShop')
section = models.ForeignKey('MSection')
revision = models.IntegerField()
def get_upload_to(self,filename):
ext = filename.split('.')[-1]
filename = u'%s.%s' % (get_datetime_string(),ext)
return os.path.join(self.default_upload_to,filename)
filepath = models.FileField(upload_to=get_upload_to)
user = models.ForeignKey('MUser',null=True,on_delete=models.SET_NULL)
upload_date = models.DateTimeField(auto_now=True,auto_now_add=True)
default_upload_to = 'excelfile'
custom django user (method 1):
class MUser(User):
objects = UserManager()
display_name = models.CharField(max_length = 24, default='')
permission_choices = (
('1','admin'),
('2','normal'),
)
permission= models.CharField(max_length= 5,choices=permission_choices)
shop = models.ForeignKey('MShop',blank=True,null=True)
def __unicode__(self):
return self.usernam
make download response
def generate_download_response(file_path,file_name):
response = HttpResponse(readFile(file_path))
response['Content-Length'] = os.path.getsize(file_path)
response['Content-type'] = 'application/vnd.ms-excel'
response['Content-Disposition'] = 'attachment; filename="%s"' % file_name.encode('cp932')
response.set_cookie('fileDownload','true',path='/')
return respons
def readFile(fn, buf_size=262144):
f = open(fn, "rb")
while True:
c = f.read(buf_size)
if c:
yield c
else:
break
f.close()
custom 404,500 view
A).config in settings.py ->debug = False
B).config in urls.py -> handler404 = ‘webapp.views.handle_404_view’ handler500 = ‘webapp.views.handler_500_view
C).config in views.py ->
def handle_500_view(request):
t = loader.get_template('500.html')
type,value,tb = sys.exc_info()
return HttpResponseServerError(t.render(RequestContext(request,{'excption_value':value})))
def handle_404_view(request):
return render(request,'404.html',status=404
formset
transactions
commands
start project
django-admin startproject projectname
start application
django-admin startapp appname
syncdb
python manage.py syncdb
inspectdb
python manage.py inspectdb > filename.py
run standalone
python manage.py runserver 0.0.0.0:portnumber
run shell
python manage.py shell
multipy database
python manage.py syncdb --database=databasename
static file serve
config in settings.py
MEDIA_ROOT = os.path.join(APP_ROOT,'..\\donki-static')
MEDIA_URL = '/excel/
config in urls.py
url(r'^excel/(?P<path>.*)$', django.views.static.serve', {'document_root':MEDIA_ROOT})
apache host
wsgi
sample virtualhost:
<VirtualHost *:80>
ServerName donki.localhost
WSGIScriptAlias / c:\sourcecode\donki\donki\wsgi.py
<Directory c:\sourcecode\donki\donki>
<Files wsgi.py>
Order deny,allow
Allow from all
</Files>
</Directory>
Alias /static/ "c:/sourcecode/donki/webapp/static/"
<Directory "c:/sourcecode/donki/webapp/static/">
Order deny,allow
Allow from all
</Directory>
ErrorLog "c:\sourcecode\donki\logs\error.log"
CustomLog "c:\sourcecode\donki\logs\access.log" common
</VirtualHost>
command:
httpd -t check #script syntax
httpd -k restart #restart appache with information
* libs
* MySQL-python
* cx_Oracle
* pywin32
* 3.x &2.x
convert column to number
def conver_column_to_number(col_name):
return reduce(lambda s,a:s*26+ord(a)-ord('A')+1,col_name,0)
quick deal with cells
range = sheet.Range('start : end')
details see in win32.py
platform:
Client:
ubuntu10.10
django1.2.7
django-pyodbc from trunk
Database Server
ms sql server 2005
winxp 32bit
install tools:
sudo apt-get install unixodbc unixodbc-dev freetds-dev tdsodbc
then modify the config files
sudo vim /etc/odbcinst.ini
[FreeTDS]
Description=TDS driver
Driver=/usr/lib/odbc/libtdsodbc.so
Setup=/usr/lib/odbc/libtdsS.so
CPTimeout=
CPReuse=
odbc.ini:
sudo vim /etc/odbc.ini
[mydsn]
Description= my dns
Driver=FreeTDS
Database=mydb
Servername=myServer
TDS_Version=8.0
Port=1433
freetds.conf: append to the last.
sudo vim /etc/freetds/freetds.conf
[myServer]
host = 192.168.3.128
port = 1433
tds version = 8.0
config finished
use tsql command to test connect:
tsql -S server_ip_address -U username -P password
install pyodbc
sudo easy_install pyodbc
if have error for gcc:
sudo apt-get install python-dev build-essential
and try it angain.
write a python script to test:
import pyodbc
conn = pyodbc.connect("DSN=mydsn;UID=username;PWD=password")
cur = conn.cursor()
cur.execute('select * from test_table')
for row in cur:
print 8 ,row
if have error in conn,use following code test first.
conn = pyodbc.connect("DRIVER={FreeTDS};Server=server_ip_address;UID=username;PWD=password;DATABASE=dbname")
if it don’t run nomarlly ,please check your config file:freetds.conf,odbc.ini,odbcinst.ini
then install django-odbc from trunk
svn co http://django-pyodbc.googlecode.com/svn/trunk/
change the settings.py file like the document said.
then your can run syncdb and runserver.
I did use django1.1 and django-pyodbc1.0.x at first,pyodbc is work right,but can’t run the django syncdb and runserver. it throws some error about settings.Time_Zone,and I still don’t know how to deal with it.
it seens have some problem use django-pyodbc from trunk too.it happens when syncdb for admin site:throws a None type can’t itera.
all settings are done,you can write a model to test it .
download from here . intall it:
$ tar -xzvf ctags-5.6.tar.gz
$ cd ctags-5.6
$ make
$ sudo make install
use it: go to directory
$ ctags -R
ctrl+] go to function definition. ctrl+t go back.
download from here.
add these in .vimrc
let Tlist_Show_One_File=1
let Tlist_Exit_OnlyWindow=1
use command with :Tlist in vim.
download from here.
add these in .vimrc
let g:winManagerWindowLayout='FileExplorer|TagList'
nmap wm :WMToggle<cr>
restart vim and type ‘wm’ in normal state
download form here.
add in .vimrc
nnoremap <silent> <F3> :Grep<CR>
then you can use F3 to search your word.
1.create ras key by using ssh-keygen
ssh-keygen -t rsa
this operation will generate two keys in your system.
2.copy it to your ssh server
cat ~/.ssh/id_rsa.pub |ssh username@ssh_server_host 'cat >> ~/.ssh/authorized_keys'
3.now you can remote to your server without password.
Install tiny-mce
sodu easy_install django-tinymce
add app in settings.py
INSTALLED_APPS = (
…
'tinymce'
…
)
if you want replace textarea tag as html editor in admin page, modify your modeladmin like this
from tinymce.widgets import TinyMCE
formfield_overrides = {
models.TextField:{'widget':TinyMCE(attrs={'cols':80,'rows':20},)},
}
class Media:
js = (
'/d-media/js/tiny_mce/tiny_mce.js',
'/d-media/js/tiny_mce/textareas.js',
)
class Meta:
model = Article
…
copy the textareas.js to your static media directory.
more adavnce settings in here
start a django project:
django-admin.py startproject project
start a app:
django-admin.py startapp app
modify settings.py file the database config struct
from os import environ
import sae.const
sae_environ = environ.get('APP_NAME','')
MYSQL_DB = 'your_database_name'
MYSQL_USER = 'your_username'
MYSQL_PASS = 'your_password'
MYSQL_HOST_M = 'your_mysql_host'
MYSQL_HOST_S = 'your_mysql_host'
MYSQL_PORT = 'your_mysql_port'
if sae_environ:
MYSQL_DB = sae.const.MYSQL_DB
MYSQL_USER = sae.const.MYSQL_USER
MYSQL_PASS = sae.const.MYSQL_PASS
MYSQL_HOST_M = sae.const.MYSQL_HOST
MYSQL_HOST_S = sae.const.MYSQL_HOST_S
MYSQL_PORT = sae.const.MYSQL_PORT
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': MYSQL_DB,
'USER': MYSQL_USER,
'PASSWORD': MYSQL_PASS,
'HOST': MYSQL_HOST_M,
'PORT': MYSQL_PORT,
}
}
don’t foget to add app to INSTALLED_APPS:
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.admin',
'project.app',
)
go to app and add models in models.py
from django.db import models
from project.settings import sae_environ
class Data(models.Models):
file_name = models.CharField(max_length=100)
file_data = models.FileField(upload_to='file')
if sae_environ:
from project.ready4sae.store import SaeStorage
file_data = models.FileField(storage=SaeStorage(domain='your_domain',app='your_app'))
then you can use python manage.py syncdb to generate the datebase and the tables
use phpmyadmin to check the database your generated and export it to sql file
now,you can do your views coding.
run the server in local to test by using
dev_server.py --mysql=username:password:mysql_host:mysql_port --storage-path=/your/path/to/storage/your_domain/your_app
your must make directory first if you will use the storage sevice for your project
if it runs okey,your can now deploy your project to the sae cloud.
change your index.wsgi file to:
import sae
import os
import sys
import django.core.handlers.wsgi
os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
application = sae.create_wsgi_app(django.core.handlers.wsgi.WSGIHandler())
log in to your sae account and go to your app,something must do:
over.