New York (CNN) -- For Muslims living in the United States, the call to prayer requires a response at least five times a day. But when the U.S. Census Bureau comes calling once every 10 years, some Arab-Americans still hesitate to return the survey.

"There's a fear factor that this information may be used against you, or used for inappropriate reasons," said Rami Nuseir, president of the American Mideast Leadership Network. He said a misunderstanding of the census translates into some Arab-Americans simply not filling out the form.