I'd like to help you deal with the guilt that has overcome you and  help you experience the depth of love and forgiveness God extends to you. I don't know what sins you have committed to cause you to feel guilty but guilt is a good problem to have when it draws us to God and to the only person who can provide forgiveness, Jesus Christ. 
Jesus Christ died for the sins of the whole world (that includes you & me), past, present and future sins.  He has paid the full penalty for those sins.  Romans 8:1 says: "There is now no condemnation for those who are in Christ Jesus".    Have you accepted Jesus Christ as the Lord and Savior of your life?   Have you repented of your sins?  Repentance means not only asking for forgiveness but having a desire to turn from sinful living. 
Do you routinely confess your sins to Him?    Jesus has paid the price once and for all for your sins. He offers to forgive our sins, and to give us the free gift of eternal life...making peace between us and His Father when we decide to trust Him as Lord and Savior of our lives. He loved us so much that He died for our sins 2,000 years ago.  
The Bible says that we are sinners: Rom 3:23-24 "for all have sinned and fall short of the glory of God," (NIV) But that God has provided Jesus as the Savior: John 3:16 says:  "For God so loved the world that he gave his one and only Son, that whoever believes in him shall not perish but have eternal life. (NIV) 
The Bible also says that we can receive Jesus Christ as our Savior and have eternal life: John 3:36: "Whoever believes in the Son has eternal life, but whoever rejects the Son will not see life, for God's wrath remains on him." (NIV)  
Read my article on faith in case you need to know more about God's plan of salvation.
The only sin God can't forgive is the sin of unbelief, because salvation through Christ (which includes forgiveness and eternal life) needs to be accepted by faith in order to be applied to one's life. 
Read the following scriptures and memorize some so that you will begin relying on God's Word and assurance regarding the forgiveness He offers you in Christ. 
Heb 10:10-14 reads:  "And by that will, we have been made holy through the sacrifice of the body of Jesus Christ once for all. 11 Day after day every priest stands and performs his religious duties; again and again he offers the same sacrifices, which can never take away sins. 12 But when this priest had offered for all time one sacrifice for sins, he sat down at the right hand of God. 13 Since that time he waits for his enemies to be made his footstool, 14 because by one sacrifice he has made perfect forever those who are being made holy." (NIV) 
There are many verses assuring us of the forgiveness we have in Christ.   Read these and ask the Lord to help you believe what He has said in His Word about forgiveness. 
Acts 10:43 All the prophets testify about him that everyone who believes in him receives forgiveness of sins through his name." 
Eph 1:7" In him(Christ) we have redemption through his blood, the forgiveness of sins, in accordance with the riches of God's grace." 
Col 1:13-14" For he has rescued us from the dominion of darkness and brought us into the kingdom of the Son he loves, in whom we have redemption, the forgiveness of sins." 
Ps 65:3" When we were overwhelmed by sins, you forgave our transgressions.' 
Ps 103:2-3" Praise the LORD, O my soul, and forget not all his benefits-- who forgives all your sins and heals all your diseases,' 
 Ps 103:10-12  "He does not treat us as our sins deserve or repay us according to our iniquities. For as high as the heavens are above the earth, so great is His love for those who fear him; as far as the east is from the west, so far has He removed our transgressions from us.' 
Isa 43:25 "I, even I, am he who blots out your transgressions, for my own sake, and remembers your sins no more." 
Acts 10:43 "All the prophets testify about him that everyone who believes in him receives forgiveness of sins through his name." 
Acts 13:38 "Therefore, my brothers, I want you to know that through Jesus the forgiveness of sins is proclaimed to you." 
Rom 4:25 He was delivered over to death for our sins and was raised to life for our justification. 
 I Jn 1:9  "If we confess our sins, He is faithful and just and will forgive us our sins and purify us from all unrighteousness." (NIV) 
Counseling with a pastor or Christian counselor and reading the Word of God can help you sort out the guilt issues you struggle with and determine whether your guilt is true or false. Remember 
that (God's) "love covers a multitude of sins" (1 Peter 4:8). 
We don't know our hearts very well, but God knows us and His forgiveness is complete and eternal because Jesus Christ has paid  the penalty for our sins and "not for our sins only, but the sins of  the whole world" (1 John 2:1-2). (NIV) 
To grow in your new relationship with Christ, attend a solid Bible teaching church to learn more and to fellowship with God's family.  You can contact the WillowCreek Association for a recommendation to a churrch near you. Read your Bible and pray everyday.
Read more about how to deal with guilt as a Christian in this article: When a Christian sins.
Read the following book: 
What a Christian Believes; An Easy to Read Guide to Understanding  by Pastor Ray Pritchard for a further understandingo o the Christian faith.   God bless you! 
Other books I recommend are: (you can order them now or check out our resources page for more selections about topics on faith):
How You Can be Sure You will Spend Eternity with God by Erwin Lutzer. 
Hope for the Troubled Heart by Billy Graham  $4.79 
What's So Amazing about Grace?  by Phillip Yancey  $13.99
Praying for you.   Lynette Hoy, NCC, LCPC