03-05-2014, 12:58 PM | #1 | ||
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
DBUpdater 2 For FOF7
LINK: http://www.younglifenorthdekalb.com/...aterv2.0.4.zip
I've finished a web-based database updater for FOF7 and I think it's release-ready. The php application opens the FOF7-generated csvs of your choosing (they must first be uploaded via FTP to the same web directory as your dbupdater.php file), reads them, updates--depending on how many csvs you upload--up to 25 database tables, and removes the csvs from your server. It is not as user-friendly as Greg's great utility was. Greg is a pro, and I'm a rank amateur. It'll likely never come close to being as user-friendly, but it works (I think.) Usage will require you to make some decisions and upload some csvs at the right times during the season. Over time, I may make it "smarter" in terms of figuring out if particular tables need to be updated, but for now, it's up to the user to upload the csvs that he wants to update. FILES INCLUDED dbupdater.php--Php file that processes the csvs and updates the database. dbcreate.sql--Creates the initial blank database. INITIAL SETUP 1. Create a blank MySQL database. 2. You'll need to run the MySQL script (dbcreate.sql) on your blank database. It will create the tables and format them in the manner DBUpdater 2.0 expects. 3. Open dbupdater.php with a text editor and enter the values for your database host (usually localhost), username, password, and database name in the appropriate slots near the beginning of the script. USAGE NOTES
Here are a few things that it will do outside of just grabbing the csvs and uploading to the database as-is...
PLEASE NOTE: If you need help, please post in this thread. I may not be available immediately, and someone else might be able to assist you with your question. Please do not PM, email, or IM me for one-on-one support on this unless I specifically request it. All unsolicited private support requests regarding this utility will be ignored.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! Last edited by Ben E Lou : 11-22-2014 at 04:57 AM. |
||
03-05-2014, 07:08 PM | #2 |
"Dutch"
Join Date: Oct 2000
Location: Tampa, FL
|
Awesome stuff. Good job, Ben!
|
03-05-2014, 08:42 PM | #3 |
High School JV
Join Date: Sep 2002
Location: GA
|
Awesome stuff Ben. Enabling the next generation of great league sites.
|
03-07-2014, 07:11 AM | #5 |
College Starter
Join Date: Dec 2000
Location: Sweden
|
Hm, I get
"SELECT week FROM fof7translate_season_weeks where stage="" Table 'hflfof7.fof7translate_season_weeks' doesn't exist" There's no create script for that table in the dbcreate.sql file from what I can see.
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! Last edited by 3ric : 03-07-2014 at 07:13 AM. |
03-07-2014, 07:14 AM | #6 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Oops. Here it is. I'll add it to the file in a bit:
Code:
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-07-2014, 07:17 AM | #7 |
College Starter
Join Date: Dec 2000
Location: Sweden
|
Thanks!
Looking good now, 2032 rows inserted into fof7_awards in 2 seconds. 194 rows inserted into fof7_active_quarterbacks in 0 seconds. 2 rows inserted into fof7_active_free_agency in 0 seconds. 78852 rows inserted into fof7_career_records in 46 seconds. 16 rows inserted into fof7_game_information in 1 seconds.
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! Last edited by 3ric : 03-07-2014 at 07:21 AM. |
03-07-2014, 07:28 AM | #8 |
Pro Starter
Join Date: Oct 2000
Location: Cary, NC
|
We'll make a programmer out of you yet!
__________________
-- Greg -- Author of various FOF utilities |
03-07-2014, 07:35 AM | #9 |
College Starter
Join Date: Dec 2000
Location: Sweden
|
Next issue...
116248 rows inserted into fof7_game_records in 79 seconds. Table 'hflfof7.fof_mappings' doesn't exist SELECT id FROM fof_mappings m WHERE m.position='QB'
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! |
03-07-2014, 07:40 AM | #10 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Ahhhhh...that's an old Stelmack table that I'm using. I overlooked that new databases wouldn't have it.
Code:
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-07-2014, 07:41 AM | #11 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
(Oh, and that table has been modified to include LS.)
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-07-2014, 07:44 AM | #12 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
2.0.1 released with scripts to create those two tables. Link in first post updated.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-07-2014, 07:47 AM | #13 |
College Starter
Join Date: Dec 2000
Location: Sweden
|
Same thing with
Table 'hflfof7.fof_positions' doesn't exist select p.positiongroup FROM fof_mappings m JOIN fof_positions p on p.position=m.id WHERE m.position='QB' I assume it should be Code:
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! Last edited by Ben E Lou : 03-07-2014 at 10:14 AM. |
03-07-2014, 08:04 AM | #14 |
College Starter
Join Date: Dec 2000
Location: Sweden
|
OK, this error is related to my data this time, the player has a nickname which breaks the SQL syntax
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'Pignose"","QB","26","0","18","5","73","205","0","0","0","0","1950","1","1","71",' at line 1 INSERT INTO fof7_player_information VALUES("3856","Plunkett","Jim","0","Jim "Pignose"","QB","26","0","18","5","73","205","0","0","0","0","1950","1","1","71","4","25","22","1971","11","31","19","0","0","0","0","10","1","1")
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! |
03-07-2014, 08:11 AM | #15 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Yup. That's the table. I will add that to the zip file later this morning. On the other issue, yeah, it isn't going to handle quotation marks within a nickname field. I would just change the nickname.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-07-2014, 08:36 AM | #16 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Have you run across any other missing tables? I'd like to include those on next upload this morning. I didn't think to test this on a brand-new database. All of the leagues that I am using it on already had FOF 2007 databases.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-07-2014, 08:40 AM | #17 |
College Starter
Join Date: Dec 2000
Location: Sweden
|
This error was a little more difficult.
Column count doesn't match value count at row 1 INSERT INTO fof7_player_information VALUES("35388","Flannery","Raymond","0","","LS","0","0","0","0","73","272","0","0","0","0","1980","10","15","37","0","0","0","0","0","0","0","0","7635","30","0","1","28") I think it's because he's a long snapper, the fault is there's a missing line in fof_positions matching values 28 and 17... trying that now
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! |
03-07-2014, 08:43 AM | #18 |
College Starter
Join Date: Dec 2000
Location: Sweden
|
Yes, that was it...
11606 rows inserted into fof7_player_information in 26 seconds. 2868 rows inserted into fof7_player_record in 4 seconds. 5733 rows inserted into fof7_player_ratings_season in 4 seconds. 0 rows inserted into fof7_standings in 0 seconds. 53656 rows inserted into fof7_playoff_game_records in 37 seconds. 908 rows inserted into fof7_rookies in 0 seconds.
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! |
03-07-2014, 08:44 AM | #19 | |
College Starter
Join Date: Dec 2000
Location: Sweden
|
Quote:
No, I'm updating as I progress through the script... should be done soon
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! |
|
03-07-2014, 08:46 AM | #20 |
College Starter
Join Date: Dec 2000
Location: Sweden
|
Twice now the script has just stopped, I assume it's a timeout preventing queries going on for a long time. I'll just resume by uploading the universe file again.
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! |
03-07-2014, 08:50 AM | #21 |
College Starter
Join Date: Dec 2000
Location: Sweden
|
98865 rows inserted into fof7_season_records in 105 seconds.
485 rows inserted into fof7_staff in 1 seconds. 160 rows inserted into fof7_staff_history in 0 seconds. 33001 rows inserted into fof7_team_game_records in 31 seconds. 33 rows inserted into fof7_team_information in 0 seconds. 7796 rows inserted into fof7_team_playoff_game_records in 7 seconds. 10921 rows inserted into fof7_team_season_records in 10 seconds. 64 rows inserted into fof7_team_statistics in 1 seconds. 993 rows inserted into fof7_team_vs_team in 0 seconds. 1839 rows inserted into fof7_transactions in 2 seconds. TOTAL SCRIPT RUN TIME: 2:37. NEXT STAGE: 2005 Reg. Season Week 4
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! |
03-07-2014, 10:14 AM | #22 | |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Quote:
On fof_positions, yeah, same thing as fof_mappings: I updated those probably within the first few days after the FOF7 release to account for long snappers. It was so long ago that I forgot that I'd done it. I'll edit your post with that script and add it to the main zip as well. (It'll still be called 2.0.1 since it's so close after the first 2.0.1.)
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
|
03-07-2014, 10:18 AM | #23 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
OK. Post edited and new 2.0.1 zip file uploaded. I should have thought to test this on a blank db before I released it, but I think we're good now. Thanks for your patience, 3ric.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-07-2014, 01:36 PM | #24 | |
College Starter
Join Date: Dec 2000
Location: Sweden
|
Quote:
It's going to vary between the individual server hosts what the timeout setting is, mine is stricter than yours and that's probably hard to tell before you've tried it - could be worth mentioning in the setup instructions. No big deal, just upload the universe file again and run the dbupdater script again.
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! |
|
03-07-2014, 07:14 PM | #25 |
"Dutch"
Join Date: Oct 2000
Location: Tampa, FL
|
So this is the current RNFL process.
1. Export data from FOF7 to my local machine 2. Using FTP, upload files to the remote server 3. RDP into remote server 4. Run SQL Load Data scripts That takes about 3 minutes so that's no big deal, but the limiting factor is that our commish (PackerFanatic) has trouble doing so from his work (and I can't run it until I get home from work either). So I'm hoping I can borrow the code you've written and tweak it for our use. I'm not sure how much tweaking that would take, honestly, but if your PHP page is easily portable to other SQL structures, this would be soooo awesome for any sort of FOF MP league. One of the reasons we had problems early on was that I couldn't get permissions to run a SQL LOAD DATA script to our host...I'm thinking this could possibly solve that issue. (I have no permission problems now with my current host since I switched us to a VPS). Last edited by Dutch : 03-07-2014 at 07:15 PM. |
03-08-2014, 03:46 AM | #26 | |
College Prospect
Join Date: Sep 2010
|
Quote:
Should be able to escape that...See: PHP: mysql_real_escape_string - Manual (Although it's deprecated, but that probably involves the least changing of code) |
|
03-09-2014, 11:28 AM | #27 | |
College Starter
Join Date: Dec 2000
Location: Sweden
|
Quote:
Single quotes are probably escaped already, it was just the double quotes that caused problems.
__________________
San Diego Chargers (HFL) - Lappland Reindeers (WOOF) - Gothenburg Giants (IHOF) Indiana: A TCY VC - year 2044 - the longest running dynasty ever on FOFC! |
|
03-16-2014, 09:33 AM | #28 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Version 2.02
Version 2.02 released.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-29-2014, 11:04 AM | #29 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Very minor update to 2.02 released. I just kept it under the 2.02 name even:
I don't know that anyone is using this actively yet other than me, but if so, you'll want to just DROP TABLE fof7translate_season_weeks and then re-run the section of the dbcreate script that creates that table. It needed one extra line to handle Front Office Bowl vs. End Of Season.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-30-2014, 07:22 PM | #30 | |
n00b
Join Date: Nov 2009
|
I just tried to set up a new league site today with v2.0.2 and I received the
Quote:
error. I don't see the new table getting set up in the dbcreate.sql file and I don't see anything that looks like it via mysql> show tables; Was maybe the code accidentally removed from v2.0.2 or am I missing something (more likely)? thanks, Mark |
|
03-30-2014, 07:52 PM | #31 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
I think I removed it in order to add the Long Snapper bits to it, then forgot to re-add. Here it is. I'll add it back to the zip later this evening.
Code:
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-30-2014, 08:07 PM | #32 |
n00b
Join Date: Nov 2009
|
awesome, I think fof_positions got left out too.
thanks for the great utility, Mark |
03-30-2014, 08:18 PM | #33 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Heh. I had two copies of dbcreate.sql sitting on my HD. I must have added the new/changed tables to the old one.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-31-2014, 06:04 AM | #34 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
FWIW, I'm setting up the GML FOF7 database pretty much from scratch this morning, so that should be a good test to make sure the dbcreate and everything else are in sync. (I checked last night and I actually had *four* local copies of dbcreate.sql and three of dbupdater.php. I've pared down to one of each now.) As a data point for those curious about run time for doing player_season, it did 8 seasons in 129 seconds on my server, so roughly 15-20 seconds per season.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-31-2014, 06:08 AM | #35 | |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
...and then writing everything else was about a minute. (I did player_season_all.csv by itself, then everything else, including transactions_all.csv, together.)
Quote:
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
|
03-31-2014, 06:10 AM | #36 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Hmmm....I'm going to need to allow for game_information_all.csv, standings_all.csv, staff_history_all.csv and team_statistics_all.csv for the initial database population. None of those require consolidating files, but renaming them to the _all to tell dbupdater to load everything rather than just the current season. Adding that now...
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
03-31-2014, 06:26 AM | #37 | ||
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Quote:
Quote:
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
||
03-31-2014, 07:29 AM | #38 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Version 2.03
Version 2.03 is up, linked in first post.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
05-08-2014, 02:47 PM | #39 |
n00b
Join Date: Nov 2010
|
Ben, I just ran this for the first time and for some reason it is only exporting the player's stats for the most current week from player_season_2013. For example, we just completed week 14 and the only stats in the player_season table now are for week 14. Is there something I am missing from the Get URL? Here is what I used http://www.s487856674.onlinehome.us/....php?year=2013
Everything else exported fine including all of the transactions for 2013. |
05-08-2014, 03:03 PM | #40 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Ah, yeah. I mentioned this for "past the first season," but for player_season it would be past the first week: rename it to player_season_all.csv and it will pick up everything. Just upload universe info with it and it will process the one file.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
05-08-2014, 03:22 PM | #41 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
On that note, it does assume that an update will be made with every game week in the regular season and playoffs, so by default it just writes the most recent week's stats. If that is a problem for anyone, I suppose it wouldn't be terribly difficult to have a check for the appropriate weeks to update, but it might be a while before I get to adding that feature.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! Last edited by Ben E Lou : 05-08-2014 at 03:23 PM. |
05-08-2014, 07:24 PM | #42 |
n00b
Join Date: Nov 2010
|
Great. I just wanted to be sure that is how you intended it to be. You may want to note on the first post to run weekly. Thanks for the awesome utility. I really appreciate it.
|
06-15-2014, 10:18 AM | #43 |
n00b
Join Date: Nov 2010
|
Error during Front Office Bowl Stage
12300,41)
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '12300,41)' at line 1 This just happened this morning during the Front Office Bowl Stage. All other stages have been working fine. |
06-15-2014, 10:30 AM | #44 |
n00b
Join Date: Nov 2010
|
Think I may have figured it out. The stageinfo table has the Front Office Bowl stage named as "End of Season", while the universe file stage is written as "Front Office Bowl". Could that be the problem? I would assume the "End of Season" comes after the "Front Office Bowl".
Last edited by RedDoby : 06-15-2014 at 10:33 AM. |
06-15-2014, 11:17 AM | #45 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Yup. I actually discovered and fixed this a while ago, but I'm guessing you have a version before I made that fix. Run this script:
DELETE FROM `fof7stageinfo` WHERE stageid=50; INSERT INTO `fof7_stageinfo` VALUES(50, 'Front Office Bowl', 26); INSERT INTO `fof7_stageinfo` VALUES(51, 'End Of Season', 27); That should take care of it.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
06-15-2014, 11:35 PM | #46 |
n00b
Join Date: Nov 2010
|
Got to love an easy fix . Thanks.
|
06-22-2014, 03:40 PM | #47 |
n00b
Join Date: Nov 2010
|
I am not able to get the Front office bowl player_season stats to insert into the player_season table. I am at the end of the season stage for 2013.
0 rows inserted into fof7_player_season in 4 seconds. TOTAL SCRIPT RUN TIME: 0:04. NEXT STAGE: 2013 End Of Season Maybe there is something else that I need to fix to get these inserted? |
10-31-2014, 03:59 AM | #48 |
n00b
Join Date: Jul 2009
|
Sorry, another question.
Why can't I find a csv for the team schedules? |
10-31-2014, 04:16 AM | #49 |
Morgado's Favorite Forum Fascist
Join Date: Oct 2000
Location: Greensboro, NC
|
Because it doesn't exist. In my opinion, that's the biggest thing missing from the FOF7 export process.
__________________
The media don't understand the kinds of problems and pressures 54 million come wit'! |
10-31-2014, 04:39 AM | #50 |
Solecismic Software
Join Date: Oct 2000
Location: Canton, OH
|
|
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Thread Tools | |
|
|