It's a theory that's being discussed in international media, and was brought up in Parliament on Tuesday - that David Coleman Headley, the American national arrested for Mumbai's 26/11 attacks- was a CIA agent who went rogue.Now, Indian intelligence sources tell NDTV that they've suspected for a while that David Coleman Headley was a CIA agent.
Indian officials say they've complained to the FBI that it should have shared information about Headley with India during Headley's last trip to India in March this year. That would have allowed India to track his movements, and to arrest him.
The FBI has confirmed that it was monitoring Headley for at least a year before he was arrested in October this year. He was already under surveillance when Mumbai's 26/11 attacks took place - attacks that Headley helped to plot and execute, according to the charges against him in a Chicago court. Many experts have pointed out that the FBI had alerted India to a strike against its hotels before 26/11 - the warning was so specific that it actually named the Taj - so it's odd that Headley's name was never passed on as a part of this warning.
In the international media, especially The Daily Beast and The New York TimesThe New York Times, there have been reports that Headley was a CIA agent who went rogue. He was possibly used by America's Drug Enforcement Administration to infiltrate Pakistan's narcotics underbelly. From there, did Headley move to the CIA? That's a story that NDTV has reported on in the last two days.
Indian investigators say that Rahul Bhatt, the son of director Mahesh Bhatt, who was befriended by Headley during his trips to Mumbai, has also said that Headley often talked about commandos and wars. This, Bhatt is reported to have told officials, made him wonder if Headley was an American spy.
What could help India's National Investigation Agency (NIA) to get some more information on Headley's real identity is tracking the money he spent. Headley seemed to have large funds at his disposal. The NIA is looking into who deposited money into Headley's bank accounts.
In the Rajya Sabha on Monday, CPM MP Brinda KaratBrinda Karat raised the issue of why Indian officials have been denied access to Headley after his arrest. A team of officials who travelled to Chicago were not allowed to question Headley. And America has since said that it's "too premature" to discuss Headley's possible extradition to India. That's an odd response, say Indian officials, given that Headley stands charged of plotting India's worst terror attacks.
Saturday, May 28, 2011
Did US Hide Truth About Headley From India? || #CIA #Espionage #Pakistan #Mumbai
Did US Hide Truth About Headley From India? || #CIA #Espionage #Pakistan #Mumbai
It's a theory that's being discussed in international media, and was brought up in Parliament on Tuesday - that David Coleman Headley, the American national arrested for Mumbai's 26/11 attacks- was a CIA agent who went rogue.Now, Indian intelligence sources tell NDTV that they've suspected for a while that David Coleman Headley was a CIA agent.
Indian officials say they've complained to the FBI that it should have shared information about Headley with India during Headley's last trip to India in March this year. That would have allowed India to track his movements, and to arrest him.
The FBI has confirmed that it was monitoring Headley for at least a year before he was arrested in October this year. He was already under surveillance when Mumbai's 26/11 attacks took place - attacks that Headley helped to plot and execute, according to the charges against him in a Chicago court. Many experts have pointed out that the FBI had alerted India to a strike against its hotels before 26/11 - the warning was so specific that it actually named the Taj - so it's odd that Headley's name was never passed on as a part of this warning.
In the international media, especially The Daily Beast and The New York TimesThe New York Times, there have been reports that Headley was a CIA agent who went rogue. He was possibly used by America's Drug Enforcement Administration to infiltrate Pakistan's narcotics underbelly. From there, did Headley move to the CIA? That's a story that NDTV has reported on in the last two days.
Indian investigators say that Rahul Bhatt, the son of director Mahesh Bhatt, who was befriended by Headley during his trips to Mumbai, has also said that Headley often talked about commandos and wars. This, Bhatt is reported to have told officials, made him wonder if Headley was an American spy.
What could help India's National Investigation Agency (NIA) to get some more information on Headley's real identity is tracking the money he spent. Headley seemed to have large funds at his disposal. The NIA is looking into who deposited money into Headley's bank accounts.
In the Rajya Sabha on Monday, CPM MP Brinda KaratBrinda Karat raised the issue of why Indian officials have been denied access to Headley after his arrest. A team of officials who travelled to Chicago were not allowed to question Headley. And America has since said that it's "too premature" to discuss Headley's possible extradition to India. That's an odd response, say Indian officials, given that Headley stands charged of plotting India's worst terror attacks.
The Headley Riddle: Rogue or Tip of an Iceberg? || #Mumbai #Pakistan #Terrorism
The Headley Riddle: Rogue or Tip of an Iceberg?
The David Coleman Headley affair is getting murkier every minute. It was reported that the visa papers of Headley and his associate Tahawwur Rana were missing from the Indian consulate in Chicago. This occasioned on Thursday conflicting statements by Foreign Minister SM Krishna and his MOS Shashi Tharoor. Krishna was still seeking clarification. Tharoor claimed that Rana’s papers had been found and the search for Headley’s papers was on. The Indian consulate in Chicago reportedly claimed that the papers were never missed and the government had access to them. Earlier on Wednesday Foreign Secretary Nirupama Rao said: “I have sought a factual report from our consulate in Chicago in this regard (of the missing papers).” She expressed satisfaction with the cooperation being extended in the 26/11 investigation by US authorities. She refused to comment on reports that Headley could be a double agent working for US authorities. “I don’t want to comment on the double agent issue. It will not be professional on my part,” she said. What’s going on?To unravel the tortuous complexities of this case it would be necessary to recall briefly Headley’s background as well as the facts so far established. On the basis of these facts some deductions may be made that give rise to questions that scream for answers. David Coleman Headley was arrested on October 3, 2009 and charged with conspiring to commit terrorist attacks outside the United States. He was allegedly involved in a plot to attack the Danish newspaper in Denmark that had published cartoons of Prophet Mohammed in 2005. The court proceedings after Headley’s arrest brought out an almost daily dose of new disclosures. American, Indian and Pakistani government officials also gave out leaks that added to these revelations. On December 7 fresh federal charges were filed against Headley alleging that he was closely involved in the planning and execution of the 26/11 attack in Mumbai. According to leaks Headley reportedly had been recruited by the U.S. Drug Enforcement Administration (DEA) in 1997 after being apprehended for smuggling heroin into the United States from Pakistan . After 9/11 he reportedly worked for the FBI as a terrorism informant. After his arrest on Oct. 3, he is reportedly again cooperating with the U.S. government.
Headley was born Daood Gilani in 1960 in America to a Pakistani father and American mother. His cousin is PRO to Pakistan’s Prime Minister Gilani. The cousins last met at a funeral in 2008. From court proceedings it emerges that Headley worked as a surveillance operative and operational planner for the Pakistan-based Lashkar-e-Taiba (LET) and Harkat-ul-Jihad e-Islami (HUJI). The U.S. government has alleged that between 2006 and 2008 Headley made five extended trips to Mumbai in order to conduct surveillance for the Mumbai attacks. If as an FBI informant Headley was briefing the agency it would explain the specific warnings the U.S. provided to India about plans to attack Mumbai hotels in September 2008. After the warning the Indian government stepped up security measures at these hotels. Oddly enough, these security measures were ceased a little before the 26/11 attacks occurred.
Headley’s surveillance activities were solo operations. According to the US security and intelligence outfit, Stratfor, solo surveillance is enormously risky. Yet Headley managed to make many video recordings and photographs in alien territory without being caught. According to Stratfor, “The only rational explanation for why Headley was not noticed while conducting his surveillance is that nobody was looking.” Or did nobody choose to look? If US intelligence agencies knew Headley was their informant and acting as their mole in LET, would they not ensure his protection? According to Indian security analyst B Raman, “Since Headley was a source of the DEA, the FBI, through the DEA, was aware of his visits to India .” Yet US authorities allowed Headley frequent visits to India without questioning even though after 9/11 security was extra vigilant about all such travel by dubious individuals. An Indian MHA official has been quoted by the media to state that the CIA concealed this fact "apparently to ensure Headley did not get exposed as a US secret agent". Subsequently a CIA spokesman has flatly stated that Headley was never a CIA agent. The easy and widely accepted explanation is that Headley was a DEA informer who became a rogue double agent. To avoid embarrassment US authorities are tight-lipped about the affair and loath to share information with their Indian counterparts. This explanation is over simple. Consider the following facts and connect the dots.
Headley had close connections with terrorist outfits in Pakistan that were complicit with rogue elements of the Pakistan establishment. Headley was an agent and informer of a US agency which explains how he could operate so boldly and travel so frequently without being questioned. But Headley’s exposure came about only because of the FBI. The information about his complicity in 26/11 also came about only because of the FBI. The CIA has disowned Headley. Yet Headley undoubtedly received logistic support from some US agency that allowed him to operate with impunity. The FBI did not stop him. But the FBI did apprehend him. So which force in the US was protecting Headley? Indian authorities either lost or pretended to lose Headley’s visa papers. Indian security mysteriously lifted the security measures to protect Mumbai hotels despite FBI warnings about imminent terrorist attack on Mumbai hotels. Why?
Headley may be a rogue double agent. That is the comforting theory. There is a much more uncomfortable possibility. Circumstantial evidence suggests that Headley could have received logistic support from rogue official sources in America , Pakistan and India . In other words he may be more than a rogue double agent. He could be serving a rogue fifth column that aids terrorism and has penetrated governments in America , India and Pakistan . This scribe has always believed that there is a transnational corporate nexus that has subverted American security and has become a global threat. Is it possible that under its shadow there has grown a parallel transnational nexus that promotes terrorism? Has this nexus become so powerful that the FBI clearly opposed to Headley and responsible for arresting him remained helpless to prevent him from planning and guiding the Mumbai terrorism? Is the embarrassment facing the US much bigger than the fact of Headley being a rogue double agent? This question cannot be lightly brushed aside. Today global terrorism has acquired the means of launching widespread and unrestricted warfare.
To Reach Afghans, Pentagon Drafts Mimes, Storytellers, Wizard of Oz | Danger Room | Wired.com
To Reach Afghans, Pentagon Drafts Mimes, Storytellers, Wizard of Oz
- By Noah Shachtman
- June 25, 2010 |
- 9:59 am |
- Categories: DarpaWatch
The Pentagon’s ultimate team of technologists is looking to looking to bridgethe cultural divide in Afghanistan — with storytelling, pantomime and the Wizard of Oz.
After nine years of conflict, the U.S. military is still having trouble finding common ground with warzone locals. One way to fill that breach, Darpa figures, is through “interactive stories.”
Counterinsurgency, in many ways, is a series of negotiations — over economic development, over security, over political power. And “negotiation,” Darpa explains in a new request for information, “is best served by a culturally-specific narrative that explains why we hold a position, how it relates to other parties, and how it affects all parties both positively and negatively.”
Darpa’s tales are part of a larger Defense Department push to get smarter about the battlefield’s “human terrain.” Teams of social scientists are now embedding with combat units; Pentagon-funded researchers are building computer models of how the societies of Afghanistan and Iraq work; military pollsters are trying to tap public opinion in the conflict zones.
To come up with its storytelling tools, Darpa’s Information Processing Technologies Office is hoping to go beyond its usual cadre of neuroscientists, artificial intelligence specialists, and gadgeteers. The agency also wants contributions from “art, literature, film, dance, games development, advertising and public relations, advertising, grass roots organizing, collective decision making or any other discipline for which the respondent can make an argument that the approach bears on this task.”
There will be some technology involved — this is Darpa, after all. But the agency is encouraging “very non-traditional approaches to this problem (e.g., a virtual tribal storyteller interacts with a human mime to produce a silent theater improvisation with audience participation).”
The “Negotiate Across Cultures” project will kick off with a two-day workshop. There, Darpa will hold a “mock competition” of potential researchers. But this won’t be a gizmo demo. Instead, the “participants will use a ‘Wizard of Oz’ approach to illustrate how each approach, if implemented, would operate on selected negotiation problems within specific socio-cultural environments. In the ‘Wizard of Oz’ approach, one or two human representatives of each participant team will listen to a problem description and manually act out the operation and structure of the tool using props they bring to the event.”
Get your props ready; the workshop is planned for late July or early August.
See Also:
- Physicist: Predicting Insurgencies Is Easy. As Long as You Dumb Down Your Wars
- Don’t Hold Your Breath for ‘Sim Afghanistan’
- Darpa Wants a Lab for Sim Afghanistans
- Darpa Plan: Turn Warzone Data into Simple Stories
- House Panel Puts the Brakes on ‘Human Terrain’ (Updated)
- ‘Human Terrain’ Chief Ousted
- Human Terrain Teams MIA in Afghanistan?
HOLY FUCK! They are using WIZARD OF OZ CIA Programming. This CAN'T be good! #ProjectMonarch dammit.
Army’s Crooning, Blogging, Gaming Brain Is The Military’s New Top Officer | Danger Room | Wired.com
Army’s Crooning, Blogging, Gaming Brain Is The Military’s New Top Officer
- By Spencer Ackerman
- May 25, 2011 |
- 6:26 pm |
- Categories: Military Life
Sorry, Gen. Cartwright. Hope you like the weather in Mons, Adm. Stavridis. You’ll be staying in Norfolk a little while longer, Gen. Odierno. Because Gen. Martin Dempsey, who’s been chief of staff of the Army for five minutes under two months, will be the next chairman of the Joint Chiefs of Staff.
This has been an odd, unusual process of selecting the next top military officer. But the AP says it’s all over, and Dempsey is President Obama’s choice for the next top military officer. This is very good news for blogging, gaming, and training foreign forces. And however Dempsey’s erstwhile rivals feel about getting passed over, the real loser is PowerPoint. Maybe counterinsurgency, too.
It’s extremely rare for someone to go from running the one of the services’ intellectual centers — the Army Training and Doctrine Command, in Dempsey’s case — to being chairman in a matter of months. It’s rarer still for an officer to be given one of the service chiefdoms for a few weeks before getting kicked upstairs.
But as we wrote in December when he got the Army nod, Dempsey is an unusual officer. He’s taken to the blogosphere to explain his thinking about the military’s future — specifically, that it should train foreign armies if it wants to avoid the next bloody land war. That needs to become “a core competency for our force in the future,” he told senators in his March confirmation hearing. He’s said to be “sick of the word ‘counterinsurgency,” since it defines the Army’s task in terms of its adversaries, rather than a positive agenda. All that is going to be music to the ears of Obama as he seeks military advice on dialing down the Iraq and Afghanistan wars and moving to an era of tight defense budgets.
Oh, and another thing he’ll encourage: social media and gaming.
He reminisced in his confirmation hearing about seeing a VHS in 1981 about how Army generals were looking to the arcades for the next generation of officers. Twitter, Facebook and online gaming are the next avenues for “leader development,” he said. Game on.
He’s not without his baggage. An Army inquiry blasted the Training and Doctrine Command under Dempsey for setting unrealistic R&D requirements that led to huge wastes of money. But that didn’t so much as slow his ascent to Army chief of staff.
Just don’t come to him with any pithy slide presentations. When he took the top Army job, he prepared a four-slide PowerPoint, titled “What I’m Like.” Bottom line up front, as they say in the Army: “You may have heard I dislike PowerPoint. It’s true.” If you think PowerPoint is a race to the intellectual bottom — or just like irony – Dempsey’s your guy.
All that and quite a singing voice, as his take on Sinatra demonstrates. His encore? Surely “My Way.”
See Also:
Darpa Plan: Turn Warzone Data into Simple Stories | Danger Room | Wired.com
Darpa Plan: Turn Warzone Data into Simple Stories
- By Noah Shachtman
- December 4, 2008 |
- 6:16 am |
- Categories: DarpaWatch
Drone feeds, informant tips, news reports, captured phone calls — sometimes, a battlefield commander gets so much information, it’s hard to make sense of it all. So the Pentagon’s far out research arm, Darpa, is looking to distill all that data into "a form that is more suitable for human consumption." Namely, a story.
The author of this tale, however, would be a series of intelligent algorithms that can pull all of this information together, tease out its underlying meanings, and put it in a narrative that’s easy to follow.
"Like people," Darpa notes in a request for information, such a story-telling system would be able to "retrieve and reuse stories to construct an appropriate interpretation of events, not because the stories have the most detail, but because they convey the aspects of a situation that are most important in determining a decision."
If it works, Darpa hopes to have this Experience-based Narrative Memory (EN-Mem) system make "complex situations… simple, understandable, and solvable." To pull it off, however, researchers will have to make big leaps in understanding how people interpret, store, and reuse information — and then program computers to learn those same lessons. No wonder En-Mem is being pushed by Darpa’s Information Processing Technology Office — the gang that’s pushing for honest-to-God artificial intelligence.
Making sense of a complex situation is like understanding a story; one must construct, impose and extract an interpretation. This interpretation weaves a commonly understood narrative into the information in a way that captures the basic interactions of characters and the dynamics of their motivations while filling in details not explicitly mentioned in the input stream. It uses story lines with which we all have experience as analogies, and it simplifies the detail in order to communicate the crucial aspects of a situation. The story lines it uses are those the decision maker should be reminded of, because they are similar to the current situation based upon what the decision maker is trying to do.
The
En-Mem story starts in February, with a workshop to review the current state of cognitive science and artificial intelligence research.[Image: TextBooksRUs]
Tags: DarpaWatchAlso on Wired.com
Army Brains: Kill PowerPoint, ‘Counterinsurgency’ | Danger Room | Wired.com
Army Brains: Kill PowerPoint, ‘Counterinsurgency’
- By Spencer Ackerman
- September 16, 2010 |
- 10:09 am |
- Categories: Army and Marines
Get us off Internet Explorer. Give us back our reconnaissance squads. Stop saying “Counterinsurgency.” And enough with the PowerPoint. Hear that, America? That’s the sound of your Army as it scrubs its systems, practices and institutions for lameness.
On Tuesday, the Army started the first in a series of monthly bull sessions about what it doesn’t do well or could do better. The process, known as Unified Quest, will culminate in April with a series of recommendations to General George Casey, the chief of staff, for internal reform on everything from combat maneuvering to homeland defense to cyberwar. I wrote on Wednesday about an internal Unified Quest debate over increasing soldiers’ skills at negotiation. But there are some others that I witnessed Tuesday that might end up becoming new Army policy.
(Alas, I am unable to quote anyone by name, in keeping with Unified Quest ground rules. Apologies.)
No More “Counterinsurgency.” As a concept within the Army, counterinsurgency is here to stay. Whatever critiques can be made of the COINdinistas, the portion of the Army participating in Unified Quest this week had little appetite to purge the service’s institutional knowledge of drinking tea and protecting foreign populations. But that’s not to say the term is written in stone.
General Martin Dempsey, who heads the Army’s Training and Doctrine Command — the command that runs Unified Quest — is sick of the word, according to one of his subordinates. His beef is that the term is reactive, defining an Army task in terms of a type of enemy, rather than describing something that the Army does affirmatively. While it’s not clear what term might replace “counterinsurgency,” one of Dempsey’s subordinates told Unified Quest participants that junking it would help “avoid the false choice between COIN and major combat operations, since focusing on one point or another is going to fail, because we’re not going to get the war we expect.”Want Smart Leadership? Get Better Web Platforms And More Bandwidth. This week’s Unified Quest session is about figuring out how best to cultivate the Army’s next generation of leaders. Naturally, professional military education came up. One participant observed that the Army just doesn’t have the technological infrastructure to meet the information needs of the current crop of 18-year olds, who grew up with high-speed connections and think in terms of 140 characters at a time. Just imagine the generation after that.
One question that the opening session debated: how to get a young lieutenant to sacrifice an hour of sleep to learn how to speak Arabic through a correspondence course. No one had a good answer, but one brave soul offered a basic principle. “When it takes ten minutes to download one page” on an Army computer, she said, “it’s boring, and you don’t like it” and it sours you on the entire activity. As anyone who struggled to load Internet Explorer 6 on a terminal in a forward-deployed recreation center can attest, the military needs an IT upgrade.
Want Smart Leadership? No More PowerPoint. If this year’s Unified Quest is any indication, PowerPoint hater H.R. McMaster, a one-star general serving in Afghanistan, is going to go from slideshow dissident to prophet in a few short years. Several participants derided the Army’s over-reliance on PowerPoint openly. One recounted stories of briefings in tactical-operations centers where otherwise intelligent officers simply read PowerPoint slides out loud — even as the slides were visible on an overhead projector.
Another observed: “That’s not the way our school system is oriented” in civilian life, making it difficult for young soldiers and officers, who are used to learning in a collaborative or social environment, to ingest information through rote absorption of slides.
No one quite articulated a vision of what comes after PowerPoint. (Keynote, maybe?) And some bemoaned “Death by PowerPoint” even as they projected their own PowerPoint slides. But it only takes a few well-placed cracks in a wall to send it crashing down. There may not be many more Lawrence Sellins — the officer who became an internet celebrity for his anti-PowerPoint rant — in the Army’s future.
Modularity May Have Gone Too Far. In 2004, the Rumsfeld Pentagon instituted a massive structural change: the Army was going to center around its brigades, rather than its traditional divisions. The basic idea behind what came to be called “Modularity” is that a brigade combat team — 3500 to 5000 soldiers rather than 10-to-20,000 of them — is easier to deploy into crisis zones, easier to extract and more narrowly tailored to a specific mission. Not a bad idea in theory. But one leading participant in Unified Quest pointed out that the past five years of modularity in Iraq and Afghanistan have shown that it’s not so great in practice, as tailoring the resources necessary to accomplish open-ended missions turned into a fiasco.
The participant, a field-grade officer, observed that modularity cost brigades lots of things that a unit at war needs to succeed. Reconnaissance squads, for one thing, and engineers, for another. “Our ability to enforce security over wide areas has been one of our biggest challenges here,” he said, “and to do it without adequate reconnaissance forces?” That creates another problem. When brigades become the basis for wartime task forces, as in Iraq and Afghanistan, some of those additional needed elements — cavalrymen and engineers — get tacked on to the brigades from other units. And that creates morale and unit-cohesion problems as the brigades learn to get along with the newbies.
“We made some presumptions with modularity,” the officer elaborated to Danger Room after his presentation, and part of the task of Unified Quest is “going back and questioning some of the logic” that led up to it. Overhauling modularity would be a huge task. And abandoning it at this point is nearly unthinkable. But the purpose of the exercise is to ask big questions.
Credit: Atlantic Council
See Also:
- U.S. Officer In Afghanistan Mows Down PowerPoint Rangers
- Colonel Kicked Out of Afghanistan for Anti-PowerPoint Rant …
- Wired 11.09: PowerPoint Is Evil
- Pentagon's Craziest PowerPoint Slide Revealed
- $190 Billion for “Modular Army”
- Army Asks Itself: Shouldn't We Be Diplomats?
- HP Holds Navy Network 'Hostage' for $3.3 Billion
Next Step for Darpa’s Mind-Controlled Prosthetics: Reliability | Danger Room | Wired.com
Next Step for Darpa’s Mind-Controlled Prosthetics: Reliability
- By Madhumita Venkataramanan
- April 8, 2011 |
- 7:00 am |
- Categories: DarpaWatch
The Pentagon’s mad science division is closer than you’d think to creating thought-controlled robot limbs for a wide swath of wounded soldiers. The early experiments and prototypes have worked out. So now Darpa wants to iron out the crucial technical wrinkles—they are going to focus on fine-tuning.
The core of their new project – Reliable Central Interfaces – is prosthetic dependability. No “technology push,” no bleeding-edge scientific studies, the agency says. For this, Darpa wants to establish solid reliability.
This endeavor is the latest in a legacy of prosthetic research that reflects the Pentagon’s efforts to help injured soldiers regain limb movement. Combining scientists’ knowledge of neuroscience with cutting-edge interface technology, their aim is to decode cues from brain neurons and transmit them to limb prostheses, allowing the paralyzed to move again. And it’s very close to realization.
One of the ancestors of this technology in 2004 allowed paraplegic ex-football player Matt Nagle to move a mouse cursor, open his emails and use a TV remote control, all with his mind.
Then came the DEKA arm, the invention where nerves from the armpit were re-routed into the chest. So the “move” command went from the brain to the chest, where sensors transmitted the instruction to the bionic arm.
Another innovative leap was the Modular Prosthetic Limb; for the first time, microchips were transplanted into a human brain from where they attempted to send a move command (via a computer) to the robot arm. The ambitious technology is still a busy work-in-progress.
Although there have been some giant strides in this field, the finish line remains out of sight. And that’s where the RCI project comes in.
Now that the infrastructure is in place, Darpa wants to identify and attack the various obstacles that have cropped up at each stage of those ventures.
Their central focus: the human brain.
They’ve divided up the big problems into five main “technical areas.” One will focus on lengthening the lifespan and reliability of current micro-arrays that record communication in the brain, the keystone of the brain-to-limb project.
Another will address mechanical algorithms that decode the brain’s signals. These are essentially the brain’s translators and Darpa wants to improve their “language” skills so prosthetic limbs can understand them better.
A third area is an effort to fine-tune the sensory feedback from artificial limb to brain, to create smooth, natural movement in amputee subjects.
Darpa says that recent technical advances have shown that creating a dexterous, high-performance limb is definitely possible. But their concern is, as always, with pushing boundaries: will the technology ever be reliable enough so wounded soldiers can actually fight again? Can these soldiers ever strip and reassemble weapons on the field, with their bare robot arms?
If it works as planned, the brain-machine technology will not just be limited to amputated soldiers — Darpa has bigger plans for it. When it is perfected, it will be the single most enabling hope for people with disabilities. Linking up the brain with a computer means paralyzed people can overcome their two largest hurdles: independent control over their communication skills as well as their physical movements. Making the lame walk may sound like sci-fi. But it is tantalizingly within reach.
Photo: Johns HopkinsAlso on Wired.com
@Korgasm_ Creates FAKE RT changes URL
 K (@Korgasm_)2011-05-28 11:07 RT@ElyssaD RT @Tweet4Gary: The aim of Tweet4Gary is to get this to President Obama please help http://tr.im/v2th #freegary RT |
Friday, May 27, 2011
Tavistock's Language Project: The Origins of Newspeak
At the start of World War II, Tavistock operatives, including Brig. Gen. John Rawlings Rees in the Psychological Warfare Directorate, were busy at work on a secret language project. The target of that project was not the ``enemy,'' but the English language itself, and the English-speaking people.The Tavistock crowd had picked up on the work of British linguist C.K. Ogden, who had created a simplified version of the English language using some 850 basic words (650 nouns and 200 verbs), with rigid rules for their use. Called ``Basic English,'' or ``Basic'' for short, the product was ridiculed by most English-speaking intellectuals; Ogden's proposal to translate Classic literature, such as Marlowe and Shakespeare, into Basic, was rightfully attacked as an effort to trivialize the greatest expressions of English-language culture.
But in the bowels of the psywar directorate, the concepts behind Basic were key to large-scale control of dangerous ``thought.'' A simplified English language limits the degrees of freedom of expression, and inhibits the transmission of meaning through metaphor. (For a more detailed discussion of language and metaphor, see Lyndon LaRouche, ``On the Subject of Metaphor,'' Fidelio, Fall 1992. It is then easy to create a ``reality'' that can be shaped through the mass media, such as radio. A reduced language is a straitjacket for the human mind.
The British Ministry of Information, which controlled all broadcasting and news dissemination, decided to experiment with the effectiveness of BASIC was asked to produce was asked to produce some newscasts in Basic, which were broadcast in a number of foreign sections of the BBC, including the Indian Section, which included among its operatives 1984 author George Orwell and close friend Guy Burgess, who later was to be involved in Britain's biggest postwar Soviet spy scandal. The results were carefully monitored.
Those involved quickly discovered that, with some modification, the language was ideal to present a censored, edited version of the news. Since it lent itself to simple, declarative statements, it gave those statements the character of fact, even though the information being reported was heavily censored or even self-admitted propaganda.
Some historians have claimed that Orwell's ``Newspeak,'' in his 1984, is a simple parody of Basic. To the contrary: Orwell was one of the most avid supporters of the Basic concept of reduced language. What appealed to him most was its simplicity and its apparent ability to abolish ``jargon.'' He also thought that anything without real meaning, when reduced to its Basic translation, would be easily seen to be absurd. The utopian Orwell, in his letters, expressed concern over the power of the Ministry of Information (Miniform, as it was known) to control and manage the news. It was that aspect of the process, not Basic's degrading of the English language, that he parodied in 1984 with his ``Newspeak,'' controlled by Minitrue, the Ministry of Truth.
British "empires of the mind"
Following the presentation of a special report of the Ministry of Information on these findings in 1943, the Basic project was placed on ``highest priority'' in the War Cabinet, at the insistence of Prime Minister Winston Churchill. The project, now-declassified papers reveal, was to be expanded to include work in the United States. While not revealing the secret research on the psychological implications of Basic, Churchill became its cheerleader, promoting the new language as the basis for a renewed bond between Britain and its former colony, America. On Sept. 6, 1943, in a speech at Harvard University, Churchill called for ``a new Boston Tea Party,'' to overturn the English language and replace it with Basic. Telling his audience of Anglophiles that they were at the ``headstream'' of a mighty cultural sea change that would have a ``health-giving effect,'' he declared that the power to control language ``offer[s] far better prizes than taking away people's provinces or lands or grinding them down in exploitation. The empires of the future are the empires of the mind.''But the public side of the project met resistance from the British and American public, who, while not necessarily grasping the full implications of Basic, nonetheless resented being told how to speak. And there was no support forthcoming from the U.S. President, Franklin Roosevelt, who considered Basic ``silly.''
However, reports from the Ministry of Information to the special War Cabinet committee said that the language was unwieldy. Rather than overturn the English language, the reports argued, it were easier to simplify the latter's usage by example of the mass media news broadcasts. Radio newscasts, which had been made up of long descriptive commentaries before the war, took on the shorter formats that are featured today. The long sentences, often with literary overtones, gave way to shorter, more direct sentences and simple vocabulary.
Television news has adopted this linguistic style: simple direct sentences, with a very, very limited vocabulary. Television newscasts, never too informative and erudite, have become less so in recent years, as they were forcibly dumbed down. When Roone Arledge, the former head of ABC sports, took over its poorly rated news division in the mid-1970s, he demanded that news broadcasts be simplified and made easier to understand.
In a 1979 article in Washingtonian magazine, media expert and political scientist John David Barber supported Arledge's approach to the news, arguing that its language ``passes right over the head of the great lower half of the American electorate.'' He compiled a list of 31 words that he thought should be excised from a CBS news broadcast; included was the term ``political conspiracy.'' Wrote Barber, ``There is no way that [that] vocabulary can catch and hold the average high school graduate.'' Most news directors agree with that assessment: Vocabulary analysis of newscasts reveals that, other than specialized terms, names of places, and proper names, far less than Basic's 850-word vocabulary is employed. The vocabulary of non-news television is even more degraded and limited.
Recent studies have shown that the vocabulary of the average American, while not quite at the Basic level of 850 words (excluding proper nouns and specialized terms), is plunging toward that level.
- The Media Cartel That Controls What You Think, by L. Wolfe, The American Almanac,, May 5, 1997.
- The Cartelization of the Media, by Jeffrey Steinberg, The American Almanac,, May 5, 1997.
- Direct British Control of the U.S. Media, The American Almanac,, May 5, 1997.
- Brainwashing: How The British Use the Media For Mass Psychological Warfare, by L. Wolfe, The American Almanac,, May 5, 1997.
- British "Fellow Travellers" Control Major U.S. Media, by Jeffrey Steinberg, The American Almanac,, May 5, 1997.
- For Whom The Polls Toll, by L. Wolfe, The American Almanac,, May 5, 1997.
Python 俱乐部 [Python俱乐部]
Python 俱乐部
Python 应用
Python 之外
最近更新
Zen of Python
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import this The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! >>>Python俱乐部开放咯
“Why is Python Installed on my Computer?” FAQ — Python v2.7.1 documentation
“Why is Python Installed on my Computer?” FAQ¶
What is Python?¶
Python is a programming language. It’s used for many different applications. It’s used in some high schools and colleges as an introductory programming language because Python is easy to learn, but it’s also used by professional software developers at places such as Google, NASA, and Lucasfilm Ltd.
If you wish to learn more about Python, start with the Beginner’s Guide to Python.
Why is Python installed on my machine?¶
If you find Python installed on your system but don’t remember installing it, there are several possible ways it could have gotten there.
- Perhaps another user on the computer wanted to learn programming and installed it; you’ll have to figure out who’s been using the machine and might have installed it.
- A third-party application installed on the machine might have been written in Python and included a Python installation. For a home computer, the most common such application is PySol, a solitaire game that includes over 1000 different games and variations.
- Some Windows machines also have Python installed. At this writing we’re aware of computers from Hewlett-Packard and Compaq that include Python. Apparently some of HP/Compaq’s administrative tools are written in Python.
- All Apple computers running Mac OS X have Python installed; it’s included in the base installation.
Can I delete Python?¶
That depends on where Python came from.
If someone installed it deliberately, you can remove it without hurting anything. On Windows, use the Add/Remove Programs icon in the Control Panel.
If Python was installed by a third-party application, you can also remove it, but that application will no longer work. You should use that application’s uninstaller rather than removing Python directly.
If Python came with your operating system, removing it is not recommended. If you remove it, whatever tools were written in Python will no longer run, and some of them might be important to you. Reinstalling the whole system would then be required to fix things again.
HOWTO Use Python in the web — Python v2.7.1 documentation
HOWTO Use Python in the web¶
Author: Marek Kubica Abstract
This document shows how Python fits into the web. It presents some ways to integrate Python with a web server, and general practices useful for developing web sites.
Programming for the Web has become a hot topic since the rise of “Web 2.0”, which focuses on user-generated content on web sites. It has always been possible to use Python for creating web sites, but it was a rather tedious task. Therefore, many frameworks and helper tools have been created to assist developers in creating faster and more robust sites. This HOWTO describes some of the methods used to combine Python with a web server to create dynamic content. It is not meant as a complete introduction, as this topic is far too broad to be covered in one single document. However, a short overview of the most popular libraries is provided.
See also
While this HOWTO tries to give an overview of Python in the web, it cannot always be as up to date as desired. Web development in Python is rapidly moving forward, so the wiki page on Web Programming may be more in sync with recent development.
The Low-Level View¶
When a user enters a web site, their browser makes a connection to the site’s web server (this is called the request). The server looks up the file in the file system and sends it back to the user’s browser, which displays it (this is the response). This is roughly how the underlying protocol, HTTP, works.
Dynamic web sites are not based on files in the file system, but rather on programs which are run by the web server when a request comes in, and which generate the content that is returned to the user. They can do all sorts of useful things, like display the postings of a bulletin board, show your email, configure software, or just display the current time. These programs can be written in any programming language the server supports. Since most servers support Python, it is easy to use Python to create dynamic web sites.
Most HTTP servers are written in C or C++, so they cannot execute Python code directly – a bridge is needed between the server and the program. These bridges, or rather interfaces, define how programs interact with the server. There have been numerous attempts to create the best possible interface, but there are only a few worth mentioning.
Not every web server supports every interface. Many web servers only support old, now-obsolete interfaces; however, they can often be extended using third-party modules to support newer ones.
Common Gateway Interface¶
This interface, most commonly referred to as “CGI”, is the oldest, and is supported by nearly every web server out of the box. Programs using CGI to communicate with their web server need to be started by the server for every request. So, every request starts a new Python interpreter – which takes some time to start up – thus making the whole interface only usable for low load situations.
The upside of CGI is that it is simple – writing a Python program which uses CGI is a matter of about three lines of code. This simplicity comes at a price: it does very few things to help the developer.
Writing CGI programs, while still possible, is no longer recommended. With WSGI, a topic covered later in this document, it is possible to write programs that emulate CGI, so they can be run as CGI if no better option is available.
See also
The Python standard library includes some modules that are helpful for creating plain CGI programs:
- cgi – Handling of user input in CGI scripts
- cgitb – Displays nice tracebacks when errors happen in CGI applications, instead of presenting a “500 Internal Server Error” message
The Python wiki features a page on CGI scripts with some additional information about CGI in Python.
Simple script for testing CGI¶
To test whether your web server works with CGI, you can use this short and simple CGI program:
#!/usr/bin/env python # -*- coding: UTF-8 -*- # enable debugging import cgitb cgitb.enable() print "Content-Type: text/plain;charset=utf-8" print print "Hello World!"Depending on your web server configuration, you may need to save this code with a .py or .cgi extension. Additionally, this file may also need to be in a cgi-bin folder, for security reasons.
You might wonder what the cgitb line is about. This line makes it possible to display a nice traceback instead of just crashing and displaying an “Internal Server Error” in the user’s browser. This is useful for debugging, but it might risk exposing some confidential data to the user. You should not use cgitb in production code for this reason. You should always catch exceptions, and display proper error pages – end-users don’t like to see nondescript “Internal Server Errors” in their browsers.
Setting up CGI on your own server¶
If you don’t have your own web server, this does not apply to you. You can check whether it works as-is, and if not you will need to talk to the administrator of your web server. If it is a big host, you can try filing a ticket asking for Python support.
If you are your own administrator or want to set up CGI for testing purposes on your own computers, you have to configure it by yourself. There is no single way to configure CGI, as there are many web servers with different configuration options. Currently the most widely used free web server is Apache HTTPd, or Apache for short. Apache can be easily installed on nearly every system using the system’s package management tool. lighttpd is another alternative and is said to have better performance. On many systems this server can also be installed using the package management tool, so manually compiling the web server may not be needed.
- On Apache you can take a look at the Dynamic Content with CGI tutorial, where everything is described. Most of the time it is enough just to set +ExecCGI. The tutorial also describes the most common gotchas that might arise.
- On lighttpd you need to use the CGI module, which can be configured in a straightforward way. It boils down to setting cgi.assign properly.
Common problems with CGI scripts¶
Using CGI sometimes leads to small annoyances while trying to get these scripts to run. Sometimes a seemingly correct script does not work as expected, the cause being some small hidden problem that’s difficult to spot.
Some of these potential problems are:
- The Python script is not marked as executable. When CGI scripts are not executable most web servers will let the user download it, instead of running it and sending the output to the user. For CGI scripts to run properly on Unix-like operating systems, the +x bit needs to be set. Using chmod a+x your_script.py may solve this problem.
- On a Unix-like system, The line endings in the program file must be Unix style line endings. This is important because the web server checks the first line of the script (called shebang) and tries to run the program specified there. It gets easily confused by Windows line endings (Carriage Return & Line Feed, also called CRLF), so you have to convert the file to Unix line endings (only Line Feed, LF). This can be done automatically by uploading the file via FTP in text mode instead of binary mode, but the preferred way is just telling your editor to save the files with Unix line endings. Most editors support this.
- Your web server must be able to read the file, and you need to make sure the permissions are correct. On unix-like systems, the server often runs as user and group www-data, so it might be worth a try to change the file ownership, or making the file world readable by using chmod a+r your_script.py.
- The web server must know that the file you’re trying to access is a CGI script. Check the configuration of your web server, as it may be configured to expect a specific file extension for CGI scripts.
- On Unix-like systems, the path to the interpreter in the shebang (#!/usr/bin/env python) must be correct. This line calls /usr/bin/env to find Python, but it will fail if there is no /usr/bin/env, or if Python is not in the web server’s path. If you know where your Python is installed, you can also use that full path. The commands whereis python and type -p python could help you find where it is installed. Once you know the path, you can change the shebang accordingly: #!/usr/bin/python.
- The file must not contain a BOM (Byte Order Mark). The BOM is meant for determining the byte order of UTF-16 and UTF-32 encodings, but some editors write this also into UTF-8 files. The BOM interferes with the shebang line, so be sure to tell your editor not to write the BOM.
- If the web server is using mod_python, mod_python may be having problems. mod_python is able to handle CGI scripts by itself, but it can also be a source of issues.
mod_python¶
People coming from PHP often find it hard to grasp how to use Python in the web. Their first thought is mostly mod_python, because they think that this is the equivalent to mod_php. Actually, there are many differences. What mod_python does is embed the interpreter into the Apache process, thus speeding up requests by not having to start a Python interpreter for each request. On the other hand, it is not “Python intermixed with HTML” in the way that PHP is often intermixed with HTML. The Python equivalent of that is a template engine. mod_python itself is much more powerful and provides more access to Apache internals. It can emulate CGI, work in a “Python Server Pages” mode (similar to JSP) which is “HTML intermingled with Python”, and it has a “Publisher” which designates one file to accept all requests and decide what to do with them.
mod_python does have some problems. Unlike the PHP interpreter, the Python interpreter uses caching when executing files, so changes to a file will require the web server to be restarted. Another problem is the basic concept – Apache starts child processes to handle the requests, and unfortunately every child process needs to load the whole Python interpreter even if it does not use it. This makes the whole web server slower. Another problem is that, because mod_python is linked against a specific version of libpython, it is not possible to switch from an older version to a newer (e.g. 2.4 to 2.5) without recompiling mod_python. mod_python is also bound to the Apache web server, so programs written for mod_python cannot easily run on other web servers.
These are the reasons why mod_python should be avoided when writing new programs. In some circumstances it still might be a good idea to use mod_python for deployment, but WSGI makes it possible to run WSGI programs under mod_python as well.
FastCGI and SCGI¶
FastCGI and SCGI try to solve the performance problem of CGI in another way. Instead of embedding the interpreter into the web server, they create long-running background processes. There is still a module in the web server which makes it possible for the web server to “speak” with the background process. As the background process is independent of the server, it can be written in any language, including Python. The language just needs to have a library which handles the communication with the webserver.
The difference between FastCGI and SCGI is very small, as SCGI is essentially just a “simpler FastCGI”. As the web server support for SCGI is limited, most people use FastCGI instead, which works the same way. Almost everything that applies to SCGI also applies to FastCGI as well, so we’ll only cover the latter.
These days, FastCGI is never used directly. Just like mod_python, it is only used for the deployment of WSGI applications.
See also
- FastCGI, SCGI, and Apache: Background and Future is a discussion on why the concept of FastCGI and SCGI is better that that of mod_python.
Setting up FastCGI¶
Each web server requires a specific module.
- Apache has both mod_fastcgi and mod_fcgid. mod_fastcgi is the original one, but it has some licensing issues, which is why it is sometimes considered non-free. mod_fcgid is a smaller, compatible alternative. One of these modules needs to be loaded by Apache.
- lighttpd ships its own FastCGI module as well as an SCGI module.
- nginx also supports FastCGI.
Once you have installed and configured the module, you can test it with the following WSGI-application:
#!/usr/bin/env python # -*- coding: UTF-8 -*- from cgi import escape import sys, os from flup.server.fcgi import WSGIServer def app(environ, start_response): start_response('200 OK', [('Content-Type', 'text/html')]) yield '<h1>FastCGI Environment</h1>' yield '<table>' for k, v in sorted(environ.items()): yield '<tr><th>%s</th><td>%s</td></tr>' % (escape(k), escape(v)) yield '</table>' WSGIServer(app).run()This is a simple WSGI application, but you need to install flup first, as flup handles the low level FastCGI access.
See also
There is some documentation on setting up Django with FastCGI, most of which can be reused for other WSGI-compliant frameworks and libraries. Only the manage.py part has to be changed, the example used here can be used instead. Django does more or less the exact same thing.
mod_wsgi¶
mod_wsgi is an attempt to get rid of the low level gateways. Given that FastCGI, SCGI, and mod_python are mostly used to deploy WSGI applications, mod_wsgi was started to directly embed WSGI applications into the Apache web server. mod_wsgi is specifically designed to host WSGI applications. It makes the deployment of WSGI applications much easier than deployment using other low level methods, which need glue code. The downside is that mod_wsgi is limited to the Apache web server; other servers would need their own implementations of mod_wsgi.
mod_wsgi supports two modes: embedded mode, in which it integrates with the Apache process, and daemon mode, which is more FastCGI-like. Unlike FastCGI, mod_wsgi handles the worker-processes by itself, which makes administration easier.
Step back: WSGI¶
WSGI has already been mentioned several times, so it has to be something important. In fact it really is, and now it is time to explain it.
The Web Server Gateway Interface, or WSGI for short, is defined in PEP 333 and is currently the best way to do Python web programming. While it is great for programmers writing frameworks, a normal web developer does not need to get in direct contact with it. When choosing a framework for web development it is a good idea to choose one which supports WSGI.
The big benefit of WSGI is the unification of the application programming interface. When your program is compatible with WSGI – which at the outer level means that the framework you are using has support for WSGI – your program can be deployed via any web server interface for which there are WSGI wrappers. You do not need to care about whether the application user uses mod_python or FastCGI or mod_wsgi – with WSGI your application will work on any gateway interface. The Python standard library contains its own WSGI server, wsgiref, which is a small web server that can be used for testing.
A really great WSGI feature is middleware. Middleware is a layer around your program which can add various functionality to it. There is quite a bit of middleware already available. For example, instead of writing your own session management (HTTP is a stateless protocol, so to associate multiple HTTP requests with a single user your application must create and manage such state via a session), you can just download middleware which does that, plug it in, and get on with coding the unique parts of your application. The same thing with compression – there is existing middleware which handles compressing your HTML using gzip to save on your server’s bandwidth. Authentication is another a problem easily solved using existing middleware.
Although WSGI may seem complex, the initial phase of learning can be very rewarding because WSGI and the associated middleware already have solutions to many problems that might arise while developing web sites.
WSGI Servers¶
The code that is used to connect to various low level gateways like CGI or mod_python is called a WSGI server. One of these servers is flup, which supports FastCGI and SCGI, as well as AJP. Some of these servers are written in Python, as flup is, but there also exist others which are written in C and can be used as drop-in replacements.
There are many servers already available, so a Python web application can be deployed nearly anywhere. This is one big advantage that Python has compared with other web technologies.
See also
A good overview of WSGI-related code can be found in the WSGI wiki, which contains an extensive list of WSGI servers which can be used by any application supporting WSGI.
You might be interested in some WSGI-supporting modules already contained in the standard library, namely:
- wsgiref – some tiny utilities and servers for WSGI
Case study: MoinMoin¶
What does WSGI give the web application developer? Let’s take a look at an application that’s been around for a while, which was written in Python without using WSGI.
One of the most widely used wiki software packages is MoinMoin. It was created in 2000, so it predates WSGI by about three years. Older versions needed separate code to run on CGI, mod_python, FastCGI and standalone.
It now includes support for WSGI. Using WSGI, it is possible to deploy MoinMoin on any WSGI compliant server, with no additional glue code. Unlike the pre-WSGI versions, this could include WSGI servers that the authors of MoinMoin know nothing about.
Model-View-Controller¶
The term MVC is often encountered in statements such as “framework foo supports MVC”. MVC is more about the overall organization of code, rather than any particular API. Many web frameworks use this model to help the developer bring structure to their program. Bigger web applications can have lots of code, so it is a good idea to have an effective structure right from the beginning. That way, even users of other frameworks (or even other languages, since MVC is not Python-specific) can easily understand the code, given that they are already familiar with the MVC structure.
MVC stands for three components:
- The model. This is the data that will be displayed and modified. In Python frameworks, this component is often represented by the classes used by an object-relational mapper.
- The view. This component’s job is to display the data of the model to the user. Typically this component is implemented via templates.
- The controller. This is the layer between the user and the model. The controller reacts to user actions (like opening some specific URL), tells the model to modify the data if necessary, and tells the view code what to display,
While one might think that MVC is a complex design pattern, in fact it is not. It is used in Python because it has turned out to be useful for creating clean, maintainable web sites.
Note
While not all Python frameworks explicitly support MVC, it is often trivial to create a web site which uses the MVC pattern by separating the data logic (the model) from the user interaction logic (the controller) and the templates (the view). That’s why it is important not to write unnecessary Python code in the templates – it works against the MVC model and creates chaos in the code base, making it harder to understand and modify.
See also
The English Wikipedia has an article about the Model-View-Controller pattern. It includes a long list of web frameworks for various programming languages.
Ingredients for Websites¶
Websites are complex constructs, so tools have been created to help web developers make their code easier to write and more maintainable. Tools like these exist for all web frameworks in all languages. Developers are not forced to use these tools, and often there is no “best” tool. It is worth learning about the available tools because they can greatly simplify the process of developing a web site.
See also
There are far more components than can be presented here. The Python wiki has a page about these components, called Web Components.
Templates¶
Mixing of HTML and Python code is made possible by a few libraries. While convenient at first, it leads to horribly unmaintainable code. That’s why templates exist. Templates are, in the simplest case, just HTML files with placeholders. The HTML is sent to the user’s browser after filling in the placeholders.
Python already includes two ways to build simple templates:
>>> template = "<html><body><h1>Hello %s!</h1></body></html>" >>> print template % "Reader" <html><body><h1>Hello Reader!</h1></body></html> >>> from string import Template >>> template = Template("<html><body><h1>Hello ${name}</h1></body></html>") >>> print template.substitute(dict(name='Dinsdale')) <html><body><h1>Hello Dinsdale!</h1></body></html>To generate complex HTML based on non-trivial model data, conditional and looping constructs like Python’s for and if are generally needed. Template engines support templates of this complexity.
There are a lot of template engines available for Python which can be used with or without a framework. Some of these define a plain-text programming language which is easy to learn, partly because it is limited in scope. Others use XML, and the template output is guaranteed to be always be valid XML. There are many other variations.
Some frameworks ship their own template engine or recommend one in particular. In the absence of a reason to use a different template engine, using the one provided by or recommended by the framework is a good idea.
Popular template engines include:
See also
There are many template engines competing for attention, because it is pretty easy to create them in Python. The page Templating in the wiki lists a big, ever-growing number of these. The three listed above are considered “second generation” template engines and are a good place to start.
Data persistence¶
Data persistence, while sounding very complicated, is just about storing data. This data might be the text of blog entries, the postings on a bulletin board or the text of a wiki page. There are, of course, a number of different ways to store information on a web server.
Often, relational database engines like MySQL or PostgreSQL are used because of their good performance when handling very large databases consisting of millions of entries. There is also a small database engine called SQLite, which is bundled with Python in the sqlite3 module, and which uses only one file. It has no other dependencies. For smaller sites SQLite is just enough.
Relational databases are queried using a language called SQL. Python programmers in general do not like SQL too much, as they prefer to work with objects. It is possible to save Python objects into a database using a technology called ORM (Object Relational Mapping). ORM translates all object-oriented access into SQL code under the hood, so the developer does not need to think about it. Most frameworks use ORMs, and it works quite well.
A second possibility is storing data in normal, plain text files (some times called “flat files”). This is very easy for simple sites, but can be difficult to get right if the web site is performing many updates to the stored data.
A third possibility are object oriented databases (also called “object databases”). These databases store the object data in a form that closely parallels the way the objects are structured in memory during program execution. (By contrast, ORMs store the object data as rows of data in tables and relations between those rows.) Storing the objects directly has the advantage that nearly all objects can be saved in a straightforward way, unlike in relational databases where some objects are very hard to represent.
Frameworks often give hints on which data storage method to choose. It is usually a good idea to stick to the data store recommended by the framework unless the application has special requirements better satisfied by an alternate storage mechanism.
See also
- Persistence Tools lists possibilities on how to save data in the file system. Some of these modules are part of the standard library
- Database Programming helps with choosing a method for saving data
- SQLAlchemy, the most powerful OR-Mapper for Python, and Elixir, which makes SQLAlchemy easier to use
- SQLObject, another popular OR-Mapper
- ZODB and Durus, two object oriented databases
Frameworks¶
The process of creating code to run web sites involves writing code to provide various services. The code to provide a particular service often works the same way regardless of the complexity or purpose of the web site in question. Abstracting these common solutions into reusable code produces what are called “frameworks” for web development. Perhaps the most well-known framework for web development is Ruby on Rails, but Python has its own frameworks. Some of these were partly inspired by Rails, or borrowed ideas from Rails, but many existed a long time before Rails.
Originally Python web frameworks tended to incorporate all of the services needed to develop web sites as a giant, integrated set of tools. No two web frameworks were interoperable: a program developed for one could not be deployed on a different one without considerable re-engineering work. This led to the development of “minimalist” web frameworks that provided just the tools to communicate between the Python code and the http protocol, with all other services to be added on top via separate components. Some ad hoc standards were developed that allowed for limited interoperability between frameworks, such as a standard that allowed different template engines to be used interchangeably.
Since the advent of WSGI, the Python web framework world has been evolving toward interoperability based on the WSGI standard. Now many web frameworks, whether “full stack” (providing all the tools one needs to deploy the most complex web sites) or minimalist, or anything in between, are built from collections of reusable components that can be used with more than one framework.
The majority of users will probably want to select a “full stack” framework that has an active community. These frameworks tend to be well documented, and provide the easiest path to producing a fully functional web site in minimal time.
Some notable frameworks¶
There are an incredible number of frameworks, so they cannot all be covered here. Instead we will briefly touch on some of the most popular.
Django¶
Django is a framework consisting of several tightly coupled elements which were written from scratch and work together very well. It includes an ORM which is quite powerful while being simple to use, and has a great online administration interface which makes it possible to edit the data in the database with a browser. The template engine is text-based and is designed to be usable for page designers who cannot write Python. It supports template inheritance and filters (which work like Unix pipes). Django has many handy features bundled, such as creation of RSS feeds or generic views, which make it possible to create web sites almost without writing any Python code.
It has a big, international community, the members of which have created many web sites. There are also a lot of add-on projects which extend Django’s normal functionality. This is partly due to Django’s well written online documentation and the Django book.
Note
Although Django is an MVC-style framework, it names the elements differently, which is described in the Django FAQ.
TurboGears¶
Another popular web framework for Python is TurboGears. TurboGears takes the approach of using already existing components and combining them with glue code to create a seamless experience. TurboGears gives the user flexibility in choosing components. For example the ORM and template engine can be changed to use packages different from those used by default.
The documentation can be found in the TurboGears wiki, where links to screencasts can be found. TurboGears has also an active user community which can respond to most related questions. There is also a TurboGears book published, which is a good starting point.
The newest version of TurboGears, version 2.0, moves even further in direction of WSGI support and a component-based architecture. TurboGears 2 is based on the WSGI stack of another popular component-based web framework, Pylons.
Zope¶
The Zope framework is one of the “old original” frameworks. Its current incarnation in Zope2 is a tightly integrated full-stack framework. One of its most interesting feature is its tight integration with a powerful object database called the ZODB (Zope Object Database). Because of its highly integrated nature, Zope wound up in a somewhat isolated ecosystem: code written for Zope wasn’t very usable outside of Zope, and vice-versa. To solve this problem the Zope 3 effort was started. Zope 3 re-engineers Zope as a set of more cleanly isolated components. This effort was started before the advent of the WSGI standard, but there is WSGI support for Zope 3 from the Repoze project. Zope components have many years of production use behind them, and the Zope 3 project gives access to these components to the wider Python community. There is even a separate framework based on the Zope components: Grok.
Zope is also the infrastructure used by the Plone content management system, one of the most powerful and popular content management systems available.
Other notable frameworks¶
Of course these are not the only frameworks that are available. There are many other frameworks worth mentioning.
Another framework that’s already been mentioned is Pylons. Pylons is much like TurboGears, but with an even stronger emphasis on flexibility, which comes at the cost of being more difficult to use. Nearly every component can be exchanged, which makes it necessary to use the documentation of every single component, of which there are many. Pylons builds upon Paste, an extensive set of tools which are handy for WSGI.
And that’s still not everything. The most up-to-date information can always be found in the Python wiki.
See also
The Python wiki contains an extensive list of web frameworks.
Most frameworks also have their own mailing lists and IRC channels, look out for these on the projects’ web sites. There is also a general “Python in the Web” IRC channel on freenode called #python.web.